-- Logs begin at Wed 2022-08-24 11:40:42 UTC, end at Wed 2022-08-24 13:35:10 UTC. -- Aug 24 12:10:38.388701 dv-cinder-848899-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 12:10:38.397005 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 12:10:38 2022] *** Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: nodename: dv-cinder-848899-3-iscsi Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: machine: x86_64 Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: clock source: unix Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: pcre jit disabled Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: detected number of CPU cores: 4 Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: current working directory: / Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: detected binary path: /usr/bin/uwsgi-core Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: your processes number limit is 47680 Aug 24 12:10:38.399221 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: your memory page size is 4096 bytes Aug 24 12:10:38.399856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: detected max file descriptor number: 2048 Aug 24 12:10:38.399856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: lock engine: pthread robust mutexes Aug 24 12:10:38.399856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: thunder lock: enabled Aug 24 12:10:38.399856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 12:10:38.399856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 12:10:38.438856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: Python main interpreter initialized at 0x555945a829c0 Aug 24 12:10:38.438856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: python threads support enabled Aug 24 12:10:38.438856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: your server socket listen backlog is limited to 100 connections Aug 24 12:10:38.438856 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: your mercy for graceful operations on workers is 90 seconds Aug 24 12:10:38.443013 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 12:10:38.443013 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: *** Operational MODE: preforking *** Aug 24 12:10:38.443013 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: *** uWSGI is running in multiple interpreter mode *** Aug 24 12:10:38.443013 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: spawned uWSGI master process (pid: 81667) Aug 24 12:10:38.443013 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: spawned uWSGI worker 1 (pid: 81680, cores: 1) Aug 24 12:10:38.443013 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: spawned uWSGI worker 2 (pid: 81681, cores: 1) Aug 24 12:10:38.443013 dv-cinder-848899-3-iscsi devstack@keystone.service[81667]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 12:10:38.439732 dv-cinder-848899-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 12:10:42.794738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:10:42.794738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: from cryptography.utils import int_from_bytes Aug 24 12:10:42.795527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:10:42.795527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: from cryptography.utils import int_from_bytes Aug 24 12:10:42.795930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:10:42.795930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: from cryptography.utils import int_from_bytes Aug 24 12:10:42.796039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:10:42.796039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: from cryptography.utils import int_from_bytes Aug 24 12:10:42.845911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 12:10:42.846101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 12:10:42.846188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 12:10:42.846278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] command line args: [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 12:10:42.846357 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 12:10:42.846525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 12:10:42.846615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] command line args: [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 12:10:42.846712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ================================================================================ {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 12:10:42.846836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 12:10:42.846967 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] admin_token = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ================================================================================ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 12:10:42.847213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] config_dir = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847302 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] admin_token = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config_dir = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] config_source = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847660 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] debug = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] config_source = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.847955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] debug = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.848988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] list_limit = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] log_config_append = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_config_append = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] log_dir = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_dir = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.849987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] log_file = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.850997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851117 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.851944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.852878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.853002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.853122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.853198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.853396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.853476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.853885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] publish_errors = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] publish_errors = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.854951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] standard_threads = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] standard_threads = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.855998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_journal = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_json = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] use_journal = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_stderr = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] use_json = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] use_syslog = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.856985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] use_stderr = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.857072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.857150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] use_syslog = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.857268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.857343 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:10:42.857412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.857482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.857657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.857737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.857901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.857987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.858144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.858231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.858383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.858465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.858620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.858718 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.863962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.864928 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.865957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.866973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.867998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.868884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.869882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.874972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.875922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.876962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.877965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.878933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.879966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880815 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.880969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.881970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882196 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882196 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.882951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.external = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] auth.external = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.password = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] auth.password = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] auth.token = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.883963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] auth.token = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.884882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] credential.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.885883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.886877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.887042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.887296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.887369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.887547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.887770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.887840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.888021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.888300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.888543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.888803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.889952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.890908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.891873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.892838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893717 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.893944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.894924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895624 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.895956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.896963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.897902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.898861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.899977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.900956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.901892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.902995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.903888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.904996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.905127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.905260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.905385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.905508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.905632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.905756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.905887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.906548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.906695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.906819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.906940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.907966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.908950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.909936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910718 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.910987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.911981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.912981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.913113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.913237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.913360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.913483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.913606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.913729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.913926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.914973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.915915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916173 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.916894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] role.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] role.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.917996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.918116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.918237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.918365 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.918486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.918606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.918766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.918890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.919909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.920912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.921036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.921173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.921173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.921416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.921541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.921741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.922371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.922496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.922728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.922860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.922985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.923988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.924978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.925961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.926964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.927951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.928963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929658 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.929901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.930927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.connection = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.931958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.932943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.933972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.934102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.934297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.934424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.934544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.934708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.934829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.934954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.935974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:10:42.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 12:10:42.962013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.962240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.962464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.962990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.963224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.963450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.971343 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.972406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:42.999658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.000417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.001366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.001602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.055322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.065274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.105543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.105793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.116870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.117104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.262288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.262513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.262762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.262990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.263201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.263303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.263418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.263498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.263635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.263850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.264064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.264277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.286297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.286533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.286767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.286875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.287868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.288081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.299234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.299479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.311915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.312117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.312212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.312383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.312997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81680) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.313859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.314036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81681) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:10:43.330940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.331149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:10:43.331493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.331575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.331697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:10:43.331798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:10:43.331977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.332085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.332244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.332324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:10:43.332587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.332754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.332879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.333019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.333422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.333715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.342295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.342577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.342872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.342954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.343159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.343240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.343456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.343826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.350610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.351216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.353520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.353809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.354056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.354138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.354337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.354419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.354615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.354714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.354902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.354984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.355176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.355271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.355431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.355512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.355699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.355826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.355971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.356107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.356351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.356432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.356625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.356894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.379485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.379950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.380948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.381232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.382617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.383145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.385887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.386171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.386449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.389557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.389848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.390136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.394003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.394288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.397032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.397326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.398332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.398622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.398926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.399205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.399476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.399749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.400016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.400283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.401409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.401705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.401995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.402285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.402567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.402886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.403152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.403428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.417969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.418253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.421480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.421810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.422122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.425326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.425624 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.425715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.428883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.429189 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.429780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.432901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.433369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.436180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.436477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.436576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.436759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.437052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.437377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.437668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.439784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.440094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.440384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.440674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.440957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.441249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.453773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.454056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.454321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.454730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.457559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.457849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.458118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.458382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.461575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.463120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.463397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.465264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.465974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.466257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.466548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.466808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.466899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.467088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.467177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.470342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.470630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.470950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.471248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.471537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.478867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.479149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.479424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.479700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.479978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.480253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.480527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.482761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.483050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.483332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.483617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.483901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.484459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.484757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.496000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.499004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.500044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.501892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.503069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.503330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.504787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.505065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.505354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.505933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.507391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.508865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.509150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.509447 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.511730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.513166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.516431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.516609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.516922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.517222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.517529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.517874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.520868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.521168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.521457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.521741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.525369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.525661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.525938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.529750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.530046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.530331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.533358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81681) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.533639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.533917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.534189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.534460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.534762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.535034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.535297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.535570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.535842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.536112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.536383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.536652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.536907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81681) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.537776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81680) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:10:43.538065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.538348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.538633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.538932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.539219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.539495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.539765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.540041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.540319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.540602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.540877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.541153 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.541414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81680) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:10:43.571343 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555945a829c0 pid: 81681 (default app) Aug 24 12:10:43.574688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bb60-a768-4b96-bbea-9444b14aa6c7 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:43.574975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bb60-a768-4b96-bbea-9444b14aa6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:43.575246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bb60-a768-4b96-bbea-9444b14aa6c7 None None] PATH_INFO: `/v3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:43.576066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555945a829c0 pid: 81680 (default app) Aug 24 12:10:43.579424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Wed Aug 24 12:10:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:10:50.921316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9749e-b8c3-4095-bd7c-b1e791b628f3 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:50.921831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9749e-b8c3-4095-bd7c-b1e791b628f3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:50.921914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9749e-b8c3-4095-bd7c-b1e791b628f3 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:50.926636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:10:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:10:50.933152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:50.933552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:50.933694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:51.004648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: INFO dbcounter [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] Registered counter for database keystone Aug 24 12:10:51.023570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG oslo_db.sqlalchemy.engines [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81681) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 12:10:51.029936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writer thread running {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 12:10:51.214473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG passlib.handlers.bcrypt [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81681) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 12:10:51.214843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG passlib.handlers.bcrypt [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81681) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 12:10:51.269556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:10:51.271182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:10:51.399016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:10:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 12:10:51.891896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9749e-b8c3-4095-bd7c-b1e791b628f3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:51.958506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: INFO dbcounter [None req-21e9749e-b8c3-4095-bd7c-b1e791b628f3 None None] Registered counter for database keystone Aug 24 12:10:51.979625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG oslo_db.sqlalchemy.engines [None req-21e9749e-b8c3-4095-bd7c-b1e791b628f3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81680) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 12:10:51.985982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writer thread running {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 12:10:52.102666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9749e-b8c3-4095-bd7c-b1e791b628f3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:52.106867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311eaa9c-42a0-43bb-9e63-5878448c5e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:52.107477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eaa9c-42a0-43bb-9e63-5878448c5e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:52.107752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eaa9c-42a0-43bb-9e63-5878448c5e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:52.108020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eaa9c-42a0-43bb-9e63-5878448c5e49 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:52.116228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311eaa9c-42a0-43bb-9e63-5878448c5e49 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:52.167385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING keystone.common.rbac_enforcer.enforcer [None req-311eaa9c-42a0-43bb-9e63-5878448c5e49 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 12:10:52.167670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311eaa9c-42a0-43bb-9e63-5878448c5e49 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:52.177518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Aug 24 12:10:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:10:52.183527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:52.198176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc45a4d-a0f5-4441-a27d-1a7ab1f7a815 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:52.202557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851ff62-5323-4dc8-9967-0441303dad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:52.203715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ff62-5323-4dc8-9967-0441303dad92 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:52.203999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ff62-5323-4dc8-9967-0441303dad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:52.204273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851ff62-5323-4dc8-9967-0441303dad92 admin admin] PATH_INFO: `/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:52.205843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ff62-5323-4dc8-9967-0441303dad92 admin admin] RBAC: Authorizing `identity:get_project(project_id=4cfd603abe4c44b2a5789dbc4d8a6f2b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:52.257995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1851ff62-5323-4dc8-9967-0441303dad92 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 12:10:52.258339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851ff62-5323-4dc8-9967-0441303dad92 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:52.262853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Wed Aug 24 12:10:52 2022] GET /identity/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:10:54.113201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72e08-47c3-489d-b251-beb4f460a15f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:54.113812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72e08-47c3-489d-b251-beb4f460a15f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:54.113812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f72e08-47c3-489d-b251-beb4f460a15f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:54.117927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:10:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:10:54.123334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f332cf-88e6-41de-9b41-7e14c4de506b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:54.123757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f332cf-88e6-41de-9b41-7e14c4de506b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:54.123898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f332cf-88e6-41de-9b41-7e14c4de506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:54.143235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f4f332cf-88e6-41de-9b41-7e14c4de506b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:10:54.144758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f4f332cf-88e6-41de-9b41-7e14c4de506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:10:54.152456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:10:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 12:10:54.419659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f72e08-47c3-489d-b251-beb4f460a15f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:54.432431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f72e08-47c3-489d-b251-beb4f460a15f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:54.436533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b316de7-a1ca-4534-9207-95eb434fe5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:54.437126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b316de7-a1ca-4534-9207-95eb434fe5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:54.437406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b316de7-a1ca-4534-9207-95eb434fe5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:54.437679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b316de7-a1ca-4534-9207-95eb434fe5b0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:54.445208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b316de7-a1ca-4534-9207-95eb434fe5b0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:54.447005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b316de7-a1ca-4534-9207-95eb434fe5b0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:54.456061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Aug 24 12:10:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:10:54.461357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f332cf-88e6-41de-9b41-7e14c4de506b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:54.470964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f332cf-88e6-41de-9b41-7e14c4de506b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:54.475211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06656783-70a4-4d70-8846-6f8ebcdf395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:54.475827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656783-70a4-4d70-8846-6f8ebcdf395c admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:54.476110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656783-70a4-4d70-8846-6f8ebcdf395c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:54.476384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656783-70a4-4d70-8846-6f8ebcdf395c admin admin] PATH_INFO: `/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:54.477677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06656783-70a4-4d70-8846-6f8ebcdf395c admin admin] RBAC: Authorizing `identity:get_user(user_id=ad9be8700f45469d8fb5f44fbb2eeb2c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:54.479527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06656783-70a4-4d70-8846-6f8ebcdf395c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:54.481034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Wed Aug 24 12:10:54 2022] GET /identity/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:10:56.445306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92cbd-2778-4c6e-8ebe-6663141902de None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:56.445788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92cbd-2778-4c6e-8ebe-6663141902de None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:56.445870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92cbd-2778-4c6e-8ebe-6663141902de None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:56.447253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:10:56.452776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c38d-d8c5-47ad-a0a4-9d13e0a31b89 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:56.453309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c38d-d8c5-47ad-a0a4-9d13e0a31b89 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:56.453926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c38d-d8c5-47ad-a0a4-9d13e0a31b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:56.467620 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82828407-a8f1-45c0-b271-acea78a7fecf None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:56.467897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82828407-a8f1-45c0-b271-acea78a7fecf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:56.468171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82828407-a8f1-45c0-b271-acea78a7fecf None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:56.469462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:10:56.472144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-18c9c38d-d8c5-47ad-a0a4-9d13e0a31b89 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:10:56.473612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-18c9c38d-d8c5-47ad-a0a4-9d13e0a31b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:10:56.474393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:56.474697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:56.474975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:56.517193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG passlib.handlers.bcrypt [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81680) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 12:10:56.517614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG passlib.handlers.bcrypt [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81680) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 12:10:56.553765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:10:56.562344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:10:56.564493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:10:56.661243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:10:56.822116 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c38d-d8c5-47ad-a0a4-9d13e0a31b89 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:56.845199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:10:56.923784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c38d-d8c5-47ad-a0a4-9d13e0a31b89 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:56.927954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:56.956986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:56.958323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:56.958607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:56.958911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:56.970292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:10:56.978738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:10:56.979840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:10:56.980999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:10:56.982797 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:10:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:10:56.986323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:57.031565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54cbb3-86b5-4ad7-abe6-3872dd024b23 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:57.033834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:10:57.067992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:10:57.068805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:57.069696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:57.069980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:57.070256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:57.086183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924bcaa-cc72-42b3-8a67-80f755951324 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:57.121019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:57.121620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:57.121904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:57.122179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:57.122870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:10:57.123170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:57.124553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:57.136090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 8/15] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:10:57.137031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:57.138552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:57.141592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:57.239435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbcc19-1eaf-4cc8-b5b5-d2817c10f39f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:57.240575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 8/16] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:10:57.246408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:57.249517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:10:57.272430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:57.280722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:57.280722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:57.280722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] PATH_INFO: `/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:57.280722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9be8700f45469d8fb5f44fbb2eeb2c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:57.280722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:57.280722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:10:57 2022] GET /identity/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:10:57.283730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:57.298235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:10:57.341230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e97e9c-7128-4edf-adf4-2dbad4e178c0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:57.375297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:57.376265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:57.376542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:57.376813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:57.380980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7f5d5-124a-45e1-8230-31da944b5290 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:57.394593 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:57.396068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:10:57.414383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:10:57.415303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:10:57.416092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:10:57.416092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:10:57.416092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:10:57.416250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:57.416339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 9/18] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:10:57.416403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:57.416403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:57.416511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:57.417612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:57.419192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:57.420833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 10/19] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:10:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:10:57.423008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:57.427308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:10:57.479794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:10:57.506635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:10:57.523473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bb74c-3c61-43ce-9bfa-3f6a7f9bebfd None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:57.525552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd67b8-294c-4bef-b59f-c1e25b412ad4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:10:57.557535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaaf10-abaa-4bfa-8861-cd60f74fa2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:57.558134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaaf10-abaa-4bfa-8861-cd60f74fa2ed None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:57.558414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaaf10-abaa-4bfa-8861-cd60f74fa2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:57.558710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaaf10-abaa-4bfa-8861-cd60f74fa2ed None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:57.559355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecaaf10-abaa-4bfa-8861-cd60f74fa2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:10:57.559657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecaaf10-abaa-4bfa-8861-cd60f74fa2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:57.560626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36c929-a768-4209-9d63-c0cfaeb74a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:10:57.561041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecaaf10-abaa-4bfa-8861-cd60f74fa2ed None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:57.561222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36c929-a768-4209-9d63-c0cfaeb74a90 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:57.561499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36c929-a768-4209-9d63-c0cfaeb74a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:57.561770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36c929-a768-4209-9d63-c0cfaeb74a90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:57.562488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36c929-a768-4209-9d63-c0cfaeb74a90 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.domain.id=default, user.id=ad9be8700f45469d8fb5f44fbb2eeb2c) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:10:57.562977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36c929-a768-4209-9d63-c0cfaeb74a90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:10:57.564406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36c929-a768-4209-9d63-c0cfaeb74a90 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:10:57.570980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 11/20] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Aug 24 12:10:57 2022] GET /identity/v3/role_assignments?user.id=ad9be8700f45469d8fb5f44fbb2eeb2c&scope.domain.id=default&role.id=753fd078fdac40abb65aa43057d79c62 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:10:57.580283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 10/21] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:10:59.616552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9684cb-8af0-4806-99e0-414201157262 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:59.617548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9684cb-8af0-4806-99e0-414201157262 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:59.618025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9684cb-8af0-4806-99e0-414201157262 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:59.619848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 12/22] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:10:59.632096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00170608-eb93-468d-a2cd-011220b94d4b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:59.632979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00170608-eb93-468d-a2cd-011220b94d4b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:59.633444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00170608-eb93-468d-a2cd-011220b94d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:59.663647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-00170608-eb93-468d-a2cd-011220b94d4b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:10:59.665195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-00170608-eb93-468d-a2cd-011220b94d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:10:59.736095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 11/23] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:10:59.926351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2dfa-8d98-4730-a68b-e75973d2b6cb None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:59.926729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2dfa-8d98-4730-a68b-e75973d2b6cb None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:59.926955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb2dfa-8d98-4730-a68b-e75973d2b6cb None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:59.928719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 13/24] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:10:59.944519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e259db2-357c-465d-a902-ce333f46ad01 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:10:59.945078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e259db2-357c-465d-a902-ce333f46ad01 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:10:59.945544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e259db2-357c-465d-a902-ce333f46ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:10:59.971566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9e259db2-357c-465d-a902-ce333f46ad01 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:10:59.973029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9e259db2-357c-465d-a902-ce333f46ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:00.020832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:00.049410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 12/25] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:00.113194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c618d42-cfcf-4c6d-845a-18cd238c7cf6 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.113836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c618d42-cfcf-4c6d-845a-18cd238c7cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.114323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c618d42-cfcf-4c6d-845a-18cd238c7cf6 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.116197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 14/26] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:00.120941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a21f03-829b-4def-858f-f2b83004bf77 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.121230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a21f03-829b-4def-858f-f2b83004bf77 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.121504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a21f03-829b-4def-858f-f2b83004bf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.143685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-62a21f03-829b-4def-858f-f2b83004bf77 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:00.144886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-62a21f03-829b-4def-858f-f2b83004bf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:00.244152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 13/27] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:00.270407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c618d42-cfcf-4c6d-845a-18cd238c7cf6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:00.327608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad55dd-4388-41dd-aef1-8e40909693de None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.327891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad55dd-4388-41dd-aef1-8e40909693de None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.328126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad55dd-4388-41dd-aef1-8e40909693de None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.329455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 14/28] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:00.334356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3326403-eb35-4cdd-8e15-ab68943a2873 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.334641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3326403-eb35-4cdd-8e15-ab68943a2873 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.334953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3326403-eb35-4cdd-8e15-ab68943a2873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.375190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d3326403-eb35-4cdd-8e15-ab68943a2873 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:00.376397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d3326403-eb35-4cdd-8e15-ab68943a2873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:00.397079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c618d42-cfcf-4c6d-845a-18cd238c7cf6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:00.443417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:00.444492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.444997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.445797 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.480284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:00.486927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:00.487769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:00.488514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:00.491405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 15/29] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:00.493077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:00.503096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:00.516273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 15/30] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:00.517178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3326403-eb35-4cdd-8e15-ab68943a2873 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:00.599089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:00.688960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0253-e81d-484a-801b-4bfedfbfc309 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:00.726190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:00.726851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.727132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.727405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.740892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:00.742435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:11:00.758311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:00.759203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:11:00.760000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:11:00.760000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:11:00.760000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:00.760643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 16/31] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:00.761094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:00.768544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3326403-eb35-4cdd-8e15-ab68943a2873 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:00.818520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:00.819156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.819434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.819704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.820368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:00.820667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:00.822042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:00.834961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 16/32] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:00.835770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:00.851079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:00.899520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028fba0-8c1b-4960-ae2b-187b85d41b28 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:00.944815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:00.945432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.945717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.945988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.947188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:00.948861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:00.956198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:00.960846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65ff5-6bc7-48ab-95a2-644709c5c791 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:00.997013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:00.997661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:00.997940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:00.998286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:00.998937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:00.999234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:01.001188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:01.018543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 17/33] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:01.018543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:01.128968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da7473-7681-4137-b9f4-57ea63da9aea None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:01.147921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 17/34] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 12:11:00 2022] POST /identity/v3/roles => generated 225 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:01.156017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:01.166312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:01.167235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.167659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.168051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] PATH_INFO: `/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.169839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] RBAC: Authorizing `identity:get_user(user_id=ad9be8700f45469d8fb5f44fbb2eeb2c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:01.171516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:01.174369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 18/35] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:01 2022] GET /identity/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:01.175376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:01.210341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:01.300932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3586f1d-57e7-4410-bb95-fdaa5e02167f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:01.343569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c4ec0-12b0-473f-a6a3-a908ade3758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:01.344467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c4ec0-12b0-473f-a6a3-a908ade3758b None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.344885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c4ec0-12b0-473f-a6a3-a908ade3758b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.345294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c4ec0-12b0-473f-a6a3-a908ade3758b None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.346090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c4ec0-12b0-473f-a6a3-a908ade3758b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:01.347752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c4ec0-12b0-473f-a6a3-a908ade3758b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:01.393260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc4912-6de4-431f-9629-7cb1358c65bf None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:01.410217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:01.441816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:01.442437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.443022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.443022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.445866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:01.445866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:01.553779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 18/36] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 12:11:01 2022] POST /identity/v3/roles => generated 219 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:01.555120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:01.591293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 19/37] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:11:01 2022] POST /identity/v3/projects => generated 301 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:01.592914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797b3a-4ad9-4493-ba5a-9d04d11649d3 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.593208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797b3a-4ad9-4493-ba5a-9d04d11649d3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.594018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797b3a-4ad9-4493-ba5a-9d04d11649d3 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.595404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 20/38] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:01.596569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967de63a-ac74-4136-a9b6-a1a40366480c None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.596863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967de63a-ac74-4136-a9b6-a1a40366480c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.597159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967de63a-ac74-4136-a9b6-a1a40366480c None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.598755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 21/39] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:01.604498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e3959-cbfa-4475-89d3-fbadfe6404fa None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.604817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e3959-cbfa-4475-89d3-fbadfe6404fa None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.605068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e3959-cbfa-4475-89d3-fbadfe6404fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.632436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-617e3959-cbfa-4475-89d3-fbadfe6404fa None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:01.634282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-617e3959-cbfa-4475-89d3-fbadfe6404fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:01.695845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d4874-8ffe-456b-9eff-4e821f7eda9d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:01.716013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 22/40] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:01.717309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899b967-5206-4ef0-8cf7-c4c3cc8ac952 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.717600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899b967-5206-4ef0-8cf7-c4c3cc8ac952 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.717874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899b967-5206-4ef0-8cf7-c4c3cc8ac952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.736982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:01.737685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.737912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.738190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.738635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7899b967-5206-4ef0-8cf7-c4c3cc8ac952 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:01.739469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:01.739874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7899b967-5206-4ef0-8cf7-c4c3cc8ac952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:01.740936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:01.742188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:01.742937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 19/41] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:01.751817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:01.828244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:01.838599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 23/42] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:01.895635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4ee49-cbf8-42f2-b375-d291318452dd None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:01.934992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:01.935692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:01.935974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:01.936254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] PATH_INFO: `/v3/domains/default/users/ad9be8700f45469d8fb5f44fbb2eeb2c/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:01.938364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ad9be8700f45469d8fb5f44fbb2eeb2c, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:01.939857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:01.999756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899b967-5206-4ef0-8cf7-c4c3cc8ac952 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:02.026243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 20/43] 10.222.0.45 () {58 vars in 1361 bytes} [Wed Aug 24 12:11:01 2022] PUT /identity/v3/domains/default/users/ad9be8700f45469d8fb5f44fbb2eeb2c/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:02.109526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899b967-5206-4ef0-8cf7-c4c3cc8ac952 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:02.116817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:02.147520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:02.148142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:02.148421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:02.148691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:02.149470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:02.151183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:02.221623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:02.226582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d2c8b-0d76-458e-855d-9ebe2249852f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:02.227736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 24/44] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 12:11:01 2022] POST /identity/v3/roles => generated 223 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:11:02.229121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:02.267478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41639d-7dfd-4100-87a3-7f2daec2e076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:02.268039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41639d-7dfd-4100-87a3-7f2daec2e076 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:02.268317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41639d-7dfd-4100-87a3-7f2daec2e076 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:02.268586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41639d-7dfd-4100-87a3-7f2daec2e076 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:02.269711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41639d-7dfd-4100-87a3-7f2daec2e076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:02.271464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41639d-7dfd-4100-87a3-7f2daec2e076 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:02.273551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 21/45] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:02.282695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:02.393962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98433a72-c29e-4d8a-af71-78b35d3d7cf1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:02.432627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171eb59d-fbea-42cc-b71f-96a69965a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:02.433248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eb59d-fbea-42cc-b71f-96a69965a1c4 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:02.433663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eb59d-fbea-42cc-b71f-96a69965a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:02.433931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eb59d-fbea-42cc-b71f-96a69965a1c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:02.434594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171eb59d-fbea-42cc-b71f-96a69965a1c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:02.436083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171eb59d-fbea-42cc-b71f-96a69965a1c4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:02.519701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 25/46] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:11:02 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:03.938829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedb2a3-a005-4919-8498-9092751ee63a None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:03.939338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedb2a3-a005-4919-8498-9092751ee63a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:03.939421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedb2a3-a005-4919-8498-9092751ee63a None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:03.940700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 22/47] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:03.946246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f306b-7e7f-4bd3-a1cd-2cd33f0f9d0e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:03.946640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f306b-7e7f-4bd3-a1cd-2cd33f0f9d0e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:03.946835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f306b-7e7f-4bd3-a1cd-2cd33f0f9d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:03.965073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-130f306b-7e7f-4bd3-a1cd-2cd33f0f9d0e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:03.966241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-130f306b-7e7f-4bd3-a1cd-2cd33f0f9d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:04.010991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:04.031354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 26/48] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:04.301104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedb2a3-a005-4919-8498-9092751ee63a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:04.399595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedb2a3-a005-4919-8498-9092751ee63a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:04.416509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc5cee-5bd9-49d5-ac81-703f6f31de0c None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:04.416796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc5cee-5bd9-49d5-ac81-703f6f31de0c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:04.417076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc5cee-5bd9-49d5-ac81-703f6f31de0c None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:04.418449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 27/49] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:04.423438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c542d-7f15-4e6e-8403-e346d66df8e7 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:04.423731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c542d-7f15-4e6e-8403-e346d66df8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:04.424007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c542d-7f15-4e6e-8403-e346d66df8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:04.438309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:04.438960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:04.439239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:04.439510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:04.442319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-682c542d-7f15-4e6e-8403-e346d66df8e7 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:04.443542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-682c542d-7f15-4e6e-8403-e346d66df8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:04.449315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:04.454565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:04.455439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:04.456272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 23/50] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:04.461313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:04.492614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:04.512229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 28/51] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:04.672376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7653735-0391-4001-ac22-02e01232bc95 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:04.704783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:04.705389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:04.705672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:04.705947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:04.706612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:04.706944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:04.708329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:04.716613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 24/52] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:04.722286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c542d-7f15-4e6e-8403-e346d66df8e7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:04.793796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:04.819771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c542d-7f15-4e6e-8403-e346d66df8e7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:04.830913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:04.853710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:04.854339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:04.854622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:04.854925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] PATH_INFO: `/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:04.856032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9be8700f45469d8fb5f44fbb2eeb2c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:04.857468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:04.859441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 29/53] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:04 2022] GET /identity/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:04.865014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:04.885304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:04.892283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a932-59d2-4d5d-ace6-7a52dab1c9d9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:04.928495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:04.929054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:04.929340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:04.929712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:04.930889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:04.932327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:04.933892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 25/54] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:04.940234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:04.967612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d333f-e749-40d4-bf48-6070681fc3a6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:05.002273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:05.002941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:05.003291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:05.003565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:05.004951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:05.006442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:05.008029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 30/55] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:05.013921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:05.037388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a04c6-5c8e-471f-bf1d-5a04107e6f8c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:05.059264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:05.073918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07aade-eee7-4256-8a9a-ca16b6b7f08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:05.074528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07aade-eee7-4256-8a9a-ca16b6b7f08d None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:05.074841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07aade-eee7-4256-8a9a-ca16b6b7f08d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:05.075120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07aade-eee7-4256-8a9a-ca16b6b7f08d None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:05.075762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07aade-eee7-4256-8a9a-ca16b6b7f08d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:05.077250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07aade-eee7-4256-8a9a-ca16b6b7f08d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:05.112760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b063081-8ab4-4ddd-a94b-9b2bbff22741 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:05.145714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68565df2-1289-41b7-b5fa-8279e362da53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:05.146309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565df2-1289-41b7-b5fa-8279e362da53 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:05.146592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565df2-1289-41b7-b5fa-8279e362da53 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:05.146904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565df2-1289-41b7-b5fa-8279e362da53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:05.147629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565df2-1289-41b7-b5fa-8279e362da53 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.domain.id=default, user.id=ad9be8700f45469d8fb5f44fbb2eeb2c) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:05.147936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565df2-1289-41b7-b5fa-8279e362da53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:05.149384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565df2-1289-41b7-b5fa-8279e362da53 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:05.157813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 31/56] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Aug 24 12:11:05 2022] GET /identity/v3/role_assignments?user.id=ad9be8700f45469d8fb5f44fbb2eeb2c&scope.domain.id=default&role.id=753fd078fdac40abb65aa43057d79c62 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:05.171030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 26/57] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Aug 24 12:11:04 2022] POST /identity/v3/projects => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:11:07.093968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5b27f-7caf-4190-83db-2fc52c0ad313 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:07.094508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5b27f-7caf-4190-83db-2fc52c0ad313 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:07.094508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5b27f-7caf-4190-83db-2fc52c0ad313 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:07.095861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 32/58] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:07.100926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe181779-67b2-4a29-a05b-bb3ca0b62bc1 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:07.101299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe181779-67b2-4a29-a05b-bb3ca0b62bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:07.101482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe181779-67b2-4a29-a05b-bb3ca0b62bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:07.120063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-fe181779-67b2-4a29-a05b-bb3ca0b62bc1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:07.121568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fe181779-67b2-4a29-a05b-bb3ca0b62bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:07.130755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:07.186169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 27/59] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:07.462830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5b27f-7caf-4190-83db-2fc52c0ad313 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:07.560301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5b27f-7caf-4190-83db-2fc52c0ad313 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:07.570476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:07.592981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf46ee6-032f-495a-876b-2d5104af8e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:07.593619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf46ee6-032f-495a-876b-2d5104af8e1c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:07.593899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf46ee6-032f-495a-876b-2d5104af8e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:07.594178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf46ee6-032f-495a-876b-2d5104af8e1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:07.595275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf46ee6-032f-495a-876b-2d5104af8e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:07.596751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf46ee6-032f-495a-876b-2d5104af8e1c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:07.598641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 33/60] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:07.603704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe181779-67b2-4a29-a05b-bb3ca0b62bc1 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:07.697501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe181779-67b2-4a29-a05b-bb3ca0b62bc1 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:07.729561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e514da0-7133-4240-8d4b-95ec89b7dac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:07.730151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e514da0-7133-4240-8d4b-95ec89b7dac7 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:07.730427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e514da0-7133-4240-8d4b-95ec89b7dac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:07.730720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e514da0-7133-4240-8d4b-95ec89b7dac7 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:07.731353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e514da0-7133-4240-8d4b-95ec89b7dac7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:07.733048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e514da0-7133-4240-8d4b-95ec89b7dac7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:07.837404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:07.841678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 28/61] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:11:07 2022] POST /identity/v3/users => generated 275 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:10.751884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b91a2-188c-49be-948d-74358c9d1f22 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:10.752864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b91a2-188c-49be-948d-74358c9d1f22 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:10.753335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b91a2-188c-49be-948d-74358c9d1f22 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:10.754942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 34/62] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:10.760478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f02d8f-277d-4705-be64-b8f7b3c40177 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:10.761260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f02d8f-277d-4705-be64-b8f7b3c40177 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:10.761731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f02d8f-277d-4705-be64-b8f7b3c40177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:10.783949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f7f02d8f-277d-4705-be64-b8f7b3c40177 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:10.785450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f7f02d8f-277d-4705-be64-b8f7b3c40177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:10.869986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 29/63] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:10.875807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9838471-b0cb-4ad4-afa4-786de97f3036 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:10.876324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9838471-b0cb-4ad4-afa4-786de97f3036 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:10.876893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9838471-b0cb-4ad4-afa4-786de97f3036 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:10.878495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 35/64] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:10.883620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b867bd-eeb5-41a1-be7f-c66d68498da1 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:10.884132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b867bd-eeb5-41a1-be7f-c66d68498da1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:10.884600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b867bd-eeb5-41a1-be7f-c66d68498da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:10.906486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-77b867bd-eeb5-41a1-be7f-c66d68498da1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:10.907979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-77b867bd-eeb5-41a1-be7f-c66d68498da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:10.964741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b35eca-ea81-4e05-b821-47229ca4d57b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:10.965353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b35eca-ea81-4e05-b821-47229ca4d57b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:10.965831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b35eca-ea81-4e05-b821-47229ca4d57b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:10.967433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 36/65] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:10.972758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056fd14-ffef-43ff-a292-e1550fa278bc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:10.973287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056fd14-ffef-43ff-a292-e1550fa278bc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:10.973766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056fd14-ffef-43ff-a292-e1550fa278bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:10.994272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5056fd14-ffef-43ff-a292-e1550fa278bc None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:10.995865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5056fd14-ffef-43ff-a292-e1550fa278bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:11.003615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 30/66] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:11.094978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 37/67] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:11.241911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b867bd-eeb5-41a1-be7f-c66d68498da1 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.302848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056fd14-ffef-43ff-a292-e1550fa278bc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.313194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:11.355297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b867bd-eeb5-41a1-be7f-c66d68498da1 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:11.416318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:11.417331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.417805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.418285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.453371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:11.458439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:11.459321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:11.460083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 31/68] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:11.460712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.463737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056fd14-ffef-43ff-a292-e1550fa278bc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:11.575664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963675de-f063-4b65-9432-59755f0fec6a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:11.586612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:11.597482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:11.612294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:11.612930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.613213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.613491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.619549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:11.620546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.620966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.621663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.624042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:11.628912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:11.629943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:11.630903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:11.632261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 32/69] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:11.633099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.640216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:11.648259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:11.649118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:11.649869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:11.651570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 38/70] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:11:11.656056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad5ec0-2d55-486d-a6ed-b10c0fc02830 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:11.761001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b2670-115d-4ddb-83ef-61c786e56e3d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:11.776400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:11.777001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.777283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.777559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.778383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:11.778864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:11.780256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:11.788742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 33/71] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:11.789522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.796022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb3b2a-4ed4-45bf-879c-d38797ec4387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:11.796653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb3b2a-4ed4-45bf-879c-d38797ec4387 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.796936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb3b2a-4ed4-45bf-879c-d38797ec4387 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.797212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb3b2a-4ed4-45bf-879c-d38797ec4387 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.797885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb3b2a-4ed4-45bf-879c-d38797ec4387 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:11.798191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb3b2a-4ed4-45bf-879c-d38797ec4387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:11.799612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb3b2a-4ed4-45bf-879c-d38797ec4387 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:11.807901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 39/72] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:11.812067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff79c2-1984-4873-a310-1ce84cdb4dc9 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.812587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff79c2-1984-4873-a310-1ce84cdb4dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.812696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff79c2-1984-4873-a310-1ce84cdb4dc9 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.814502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 40/73] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:11.815448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade8e4-9139-456b-b950-2619ff13b71f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.815738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade8e4-9139-456b-b950-2619ff13b71f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.816017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade8e4-9139-456b-b950-2619ff13b71f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.817211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 41/74] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:11.821220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eade8e4-9139-456b-b950-2619ff13b71f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.838897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:11.849937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:11.895775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ae321-8f92-47b4-8fb3-5bb0874f8403 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:11.919056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eade8e4-9139-456b-b950-2619ff13b71f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:11.928486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:11.929091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.929392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.929672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.930298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:11.930597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:11.932016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:11.939871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 34/75] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:11.940660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:11.953952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210238-6cfb-4bea-9b55-68037c900573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:11.954585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210238-6cfb-4bea-9b55-68037c900573 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.954901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210238-6cfb-4bea-9b55-68037c900573 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.955181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210238-6cfb-4bea-9b55-68037c900573 None admin] PATH_INFO: `/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.956312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210238-6cfb-4bea-9b55-68037c900573 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9be8700f45469d8fb5f44fbb2eeb2c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:11.957769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210238-6cfb-4bea-9b55-68037c900573 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:11.959381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 42/76] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:11.960477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5028-c098-4bed-b6e0-1306f517a8dc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:11.960765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5028-c098-4bed-b6e0-1306f517a8dc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:11.961042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5028-c098-4bed-b6e0-1306f517a8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:11.979019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-799d5028-c098-4bed-b6e0-1306f517a8dc None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:11.980216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-799d5028-c098-4bed-b6e0-1306f517a8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:12.039170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab45f7-8629-4492-8f97-bd1cd089b420 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.041500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.044545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 43/77] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:12.045827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b123bd-1a5d-4bdb-ae60-dff16ef12c66 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.046148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b123bd-1a5d-4bdb-ae60-dff16ef12c66 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.046422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b123bd-1a5d-4bdb-ae60-dff16ef12c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.070938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e4b123bd-1a5d-4bdb-ae60-dff16ef12c66 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:12.072165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e4b123bd-1a5d-4bdb-ae60-dff16ef12c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:12.080046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.080737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.081025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.081304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.095674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:12.097065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:12.097423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.104697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 35/78] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:11 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:12.104697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.146965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 44/79] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:12.148111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b123bd-1a5d-4bdb-ae60-dff16ef12c66 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.235734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389d3b-d12c-4188-a83b-6b2bc24e6330 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.256960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b123bd-1a5d-4bdb-ae60-dff16ef12c66 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.283716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.284916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.285805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.286559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.288174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:12.290636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:12.292918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 36/80] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:12.293722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.294112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.294536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.294912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.295260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.324640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.330045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.370292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:12.371745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:12.404521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a25b2-899a-44bd-aa79-b8d4b963a88a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.442640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.443278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.443560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.443838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.473074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 45/81] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:12.473925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.510530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:12.512043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:12.530792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.540120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.572121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0988e0b-37a4-4fae-9fe3-a6be30183ba2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.605271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.605877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.606159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.606438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.607805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 37/82] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:12.609412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.676639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:12.678060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:12.707796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55218-f96c-41f9-aa65-986b1cc1cc3c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.734188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.736391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.742081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.742736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.746314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.746314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.753632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:12.758530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:12.759523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:12.760282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:12.760859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 38/83] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:12.761132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.777553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 46/84] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:12.778947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.859367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b328a-348a-4c40-abf4-d9495ec041ef None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.878027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d82b8-1f62-4704-a75b-148102001dbf None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:12.892462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.893083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.893375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.893634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.894771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:12.896216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:12.897764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 39/85] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:12.898517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.911975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:12.912606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:12.912888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:12.913167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:12.914296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:12.915779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:12.917420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 47/86] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:12.918184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:12.944504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:12.960042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:13.001162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a1437-9e50-416e-8b07-898d185b5da5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.016430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368505d-dc71-4ca9-80dc-f42f2d9ae0ba None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.035231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.035883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.036174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.036449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.037585 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.039056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.040692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 40/87] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:13.041463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:13.054962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.055571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.055937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.056126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.056794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:13.057098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.058485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.067076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 48/88] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:13.067842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:13.142341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c579eb-2d4b-4950-9778-cafcc70d8f0c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.154735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:13.165955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f37d5a-2fc8-4fc2-b54e-6f00a57b0cf1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.175369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:13.176133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.176759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.177041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.177315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.178438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.179940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.181580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 41/89] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:13 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:13.182348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:13.207606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.207606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.207606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.207606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.208003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.208003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.279319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcff6a-eea5-46f6-872e-f4092b1c1506 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.292981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 49/90] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:11:13 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:13.293985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:13.321149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.321782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.322152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.322475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.323211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=ad9be8700f45469d8fb5f44fbb2eeb2c) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:13.323525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.325059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.331967 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 42/91] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:13 2022] GET /identity/v3/role_assignments?user.id=ad9be8700f45469d8fb5f44fbb2eeb2c&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:13.332613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:13.435016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:13.444248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447cad2-0222-4dee-b928-e381118fadde None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.491960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21c019-894e-42fb-b453-1a562c0b236a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.494689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.494689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.494689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.494689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.496057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] RBAC: Adding query filter params (role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=796353704e344990a572797eb8638328) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:13.496376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.497860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.506315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 50/92] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:13 2022] GET /identity/v3/role_assignments?user.id=796353704e344990a572797eb8638328&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:13.506952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:13.523031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:13.538030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.539080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.539561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.539982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.541466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.543269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.546605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 43/93] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:13 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:13.552154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:13.657097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505cdce-a14e-4809-89e9-f12be9f0de8c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.692579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749afe0-e937-48ac-b0dc-1a38e0018ac8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:13.701173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.702556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.703077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.703555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.705013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=796353704e344990a572797eb8638328) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:13.705650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.707867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.715552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 51/94] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:13 2022] GET /identity/v3/role_assignments?user.id=796353704e344990a572797eb8638328&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=dbcd6da9a237441cb483965b19c21cff => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:13.721215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:13.739085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:13.740076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:13.740566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:13.741051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:13.824196 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:13.826152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:13.974539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 44/95] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:11:13.980473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:14.086758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:14.116648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad7dd0-2214-452a-8843-42ebfd192224 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:14.154324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ee0cb-9c75-47d1-87e5-33a04d6bf181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:14.154987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee0cb-9c75-47d1-87e5-33a04d6bf181 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:14.155321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee0cb-9c75-47d1-87e5-33a04d6bf181 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:14.156106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee0cb-9c75-47d1-87e5-33a04d6bf181 None admin] PATH_INFO: `/v3/projects/934753b97e9047f89a4b78277f71af05` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:14.157247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ee0cb-9c75-47d1-87e5-33a04d6bf181 None admin] RBAC: Authorizing `identity:get_project(project_id=934753b97e9047f89a4b78277f71af05)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:14.158706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ee0cb-9c75-47d1-87e5-33a04d6bf181 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:14.160520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 52/96] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:13 2022] GET /identity/v3/projects/934753b97e9047f89a4b78277f71af05 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:14.166385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:14.200337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:14.272311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9942fea-b856-456a-91f6-634f75a9dcc6 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:14.312318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f36432-1fc6-4fe7-8d15-92cdb2d7ddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:14.312934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f36432-1fc6-4fe7-8d15-92cdb2d7ddab None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:14.313214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f36432-1fc6-4fe7-8d15-92cdb2d7ddab None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:14.313490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f36432-1fc6-4fe7-8d15-92cdb2d7ddab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:14.314228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f36432-1fc6-4fe7-8d15-92cdb2d7ddab None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=934753b97e9047f89a4b78277f71af05, user.id=796353704e344990a572797eb8638328) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:14.314535 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f36432-1fc6-4fe7-8d15-92cdb2d7ddab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:14.316032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f36432-1fc6-4fe7-8d15-92cdb2d7ddab None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:14.321771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 45/97] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:14 2022] GET /identity/v3/role_assignments?user.id=796353704e344990a572797eb8638328&scope.project.id=934753b97e9047f89a4b78277f71af05&role.id=dbcd6da9a237441cb483965b19c21cff => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:15.850315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822605e7-c4ba-419b-8a97-afd5722a874a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:15.850873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822605e7-c4ba-419b-8a97-afd5722a874a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:15.850873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822605e7-c4ba-419b-8a97-afd5722a874a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:15.852496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 53/98] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:15.868549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12f069-8438-4642-85ca-f769c27bf650 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:15.868981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12f069-8438-4642-85ca-f769c27bf650 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:15.869184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12f069-8438-4642-85ca-f769c27bf650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:15.895814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-bc12f069-8438-4642-85ca-f769c27bf650 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:15.897044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-bc12f069-8438-4642-85ca-f769c27bf650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:15.938425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a6c32-ca88-4c19-80a6-77712e9141ec None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:15.938737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a6c32-ca88-4c19-80a6-77712e9141ec None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:15.939013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a6c32-ca88-4c19-80a6-77712e9141ec None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:15.940639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 54/99] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:15.959035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22334b9b-1a33-48c7-a5f0-bf3565737dc2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:15.959322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22334b9b-1a33-48c7-a5f0-bf3565737dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:15.959587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22334b9b-1a33-48c7-a5f0-bf3565737dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:15.985982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-22334b9b-1a33-48c7-a5f0-bf3565737dc2 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:15.987262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-22334b9b-1a33-48c7-a5f0-bf3565737dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:16.002782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 46/100] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:16.073458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 55/101] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:16.337528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12f069-8438-4642-85ca-f769c27bf650 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:16.364463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:16.451020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12f069-8438-4642-85ca-f769c27bf650 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:16.486208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113b20b-6d0a-42b8-9f00-d9c29f1b93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:16.487096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b20b-6d0a-42b8-9f00-d9c29f1b93df None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:16.487377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b20b-6d0a-42b8-9f00-d9c29f1b93df None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:16.487649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113b20b-6d0a-42b8-9f00-d9c29f1b93df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:16.498150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113b20b-6d0a-42b8-9f00-d9c29f1b93df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-f113b20b-6d0a-42b8-9f00-d9c29f1b93df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:16.502981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:16.503826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:16.504579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 47/102] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:16.511535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22334b9b-1a33-48c7-a5f0-bf3565737dc2 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:16.586948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e13e2-0a86-4b25-9c34-eedcb0531e92 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:16.587259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e13e2-0a86-4b25-9c34-eedcb0531e92 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:16.587505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e13e2-0a86-4b25-9c34-eedcb0531e92 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:16.589101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 48/103] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:16.598144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9b536-b7a5-4026-a2cc-31bb96ae4397 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:16.598426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9b536-b7a5-4026-a2cc-31bb96ae4397 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:16.598793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9b536-b7a5-4026-a2cc-31bb96ae4397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:16.625273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-78f9b536-b7a5-4026-a2cc-31bb96ae4397 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:16.626480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-78f9b536-b7a5-4026-a2cc-31bb96ae4397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:16.630146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:16.691599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22334b9b-1a33-48c7-a5f0-bf3565737dc2 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:16.751485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:16.752102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:16.752386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:16.752662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:16.753334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:16.753639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:16.755063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:16.760619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 49/104] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:16.761481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9b536-b7a5-4026-a2cc-31bb96ae4397 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:16.774742 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 56/105] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:16.780965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:16.784855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:16.949743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9b536-b7a5-4026-a2cc-31bb96ae4397 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:16.976901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc70ef7-1a45-4d0f-beea-81573b8cd772 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.003377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec36cbe-2590-4590-926e-db22b2c4656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.004033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36cbe-2590-4590-926e-db22b2c4656c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.004331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36cbe-2590-4590-926e-db22b2c4656c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.004610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36cbe-2590-4590-926e-db22b2c4656c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.015333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:17.019321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13daf71-a6b6-4faa-9642-cabf0cc70197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.020275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13daf71-a6b6-4faa-9642-cabf0cc70197 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.020693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13daf71-a6b6-4faa-9642-cabf0cc70197 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.021104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13daf71-a6b6-4faa-9642-cabf0cc70197 None admin] PATH_INFO: `/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.022374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13daf71-a6b6-4faa-9642-cabf0cc70197 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9be8700f45469d8fb5f44fbb2eeb2c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:17.024386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13daf71-a6b6-4faa-9642-cabf0cc70197 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:17.026243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 57/106] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:16 2022] GET /identity/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:17.027997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce06c0-38b7-4c99-8c79-cc5620c2ad71 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.028440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce06c0-38b7-4c99-8c79-cc5620c2ad71 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.028855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce06c0-38b7-4c99-8c79-cc5620c2ad71 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.031011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 58/107] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:17.037998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce06c0-38b7-4c99-8c79-cc5620c2ad71 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.038108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec36cbe-2590-4590-926e-db22b2c4656c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-4ec36cbe-2590-4590-926e-db22b2c4656c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:17.043339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:17.044374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:17.045227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 50/108] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:17.046039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c673e5-2960-446a-b05c-4a84c5145e2d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.046105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c673e5-2960-446a-b05c-4a84c5145e2d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.046370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c673e5-2960-446a-b05c-4a84c5145e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.070185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-66c673e5-2960-446a-b05c-4a84c5145e2d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:17.071536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-66c673e5-2960-446a-b05c-4a84c5145e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:17.142785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:17.147704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 51/109] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:17.148528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c673e5-2960-446a-b05c-4a84c5145e2d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.158924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce06c0-38b7-4c99-8c79-cc5620c2ad71 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.193540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049538f8-8e73-463f-ac2e-ed6d2d23c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.194451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049538f8-8e73-463f-ac2e-ed6d2d23c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.194990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049538f8-8e73-463f-ac2e-ed6d2d23c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.195467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049538f8-8e73-463f-ac2e-ed6d2d23c6f6 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.196625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049538f8-8e73-463f-ac2e-ed6d2d23c6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:17.198082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049538f8-8e73-463f-ac2e-ed6d2d23c6f6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:17.199939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 59/110] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:17.205462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104fb74-4644-4c15-ad90-a5c66b45105a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.205748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104fb74-4644-4c15-ad90-a5c66b45105a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.206025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104fb74-4644-4c15-ad90-a5c66b45105a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.207392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 60/111] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:17.207964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104fb74-4644-4c15-ad90-a5c66b45105a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.261980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c673e5-2960-446a-b05c-4a84c5145e2d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.279485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:17.303244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97932db0-aa1b-4828-88e6-394d2ebba731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.303864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97932db0-aa1b-4828-88e6-394d2ebba731 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.304151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97932db0-aa1b-4828-88e6-394d2ebba731 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.304431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97932db0-aa1b-4828-88e6-394d2ebba731 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.305104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97932db0-aa1b-4828-88e6-394d2ebba731 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:17.305428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97932db0-aa1b-4828-88e6-394d2ebba731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:17.306824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97932db0-aa1b-4828-88e6-394d2ebba731 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:17.318540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 52/112] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:17.318540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5db9b-dbac-45a6-8497-1d78855d87e7 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.318540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5db9b-dbac-45a6-8497-1d78855d87e7 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.318540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5db9b-dbac-45a6-8497-1d78855d87e7 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.318989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 53/113] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:17.322691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5db9b-dbac-45a6-8497-1d78855d87e7 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.324055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104fb74-4644-4c15-ad90-a5c66b45105a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.369396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aafbf-101f-40ac-bbb9-1dd206ce82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.370071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aafbf-101f-40ac-bbb9-1dd206ce82e8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.370352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aafbf-101f-40ac-bbb9-1dd206ce82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.370627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aafbf-101f-40ac-bbb9-1dd206ce82e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.381249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9aafbf-101f-40ac-bbb9-1dd206ce82e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-6e9aafbf-101f-40ac-bbb9-1dd206ce82e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:17.386540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:17.387441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:17.388271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 61/114] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:17.389557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e94b0a-5b73-419a-87c9-22f2de7190c7 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.389557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e94b0a-5b73-419a-87c9-22f2de7190c7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.389725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e94b0a-5b73-419a-87c9-22f2de7190c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.398920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:17.413608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-66e94b0a-5b73-419a-87c9-22f2de7190c7 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:17.414022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-66e94b0a-5b73-419a-87c9-22f2de7190c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:17.440428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5db9b-dbac-45a6-8497-1d78855d87e7 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.484468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d41cb7-5a0e-4c32-aa0a-cd187eb5be5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.485131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41cb7-5a0e-4c32-aa0a-cd187eb5be5f None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.485410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41cb7-5a0e-4c32-aa0a-cd187eb5be5f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.485677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41cb7-5a0e-4c32-aa0a-cd187eb5be5f None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/ad9be8700f45469d8fb5f44fbb2eeb2c/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.487595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d41cb7-5a0e-4c32-aa0a-cd187eb5be5f None admin] RBAC: Authorizing `identity:create_grant(project_id=c53f8195f07e4713b63a46a6129a59d1, user_id=ad9be8700f45469d8fb5f44fbb2eeb2c, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:17.489033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d41cb7-5a0e-4c32-aa0a-cd187eb5be5f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:17.492338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 62/115] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:17.493191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e94b0a-5b73-419a-87c9-22f2de7190c7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.550479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 54/116] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:17 2022] PUT /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/ad9be8700f45469d8fb5f44fbb2eeb2c/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:17.552100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674592-d39a-4412-93ee-7c36107569e1 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.552466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674592-d39a-4412-93ee-7c36107569e1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.552812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674592-d39a-4412-93ee-7c36107569e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.568800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:17.573537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-76674592-d39a-4412-93ee-7c36107569e1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:17.575192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-76674592-d39a-4412-93ee-7c36107569e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:17.630880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e94b0a-5b73-419a-87c9-22f2de7190c7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.661777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 55/117] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:17.670509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674592-d39a-4412-93ee-7c36107569e1 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.686269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.687282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.687741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.688228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.692705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:17.694520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:17.698798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 63/118] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:17.701653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.764965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:17.826804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674592-d39a-4412-93ee-7c36107569e1 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.832547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdf841-58f0-487b-a5cd-344b48fece07 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:17.862264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:17.868493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.869349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.869796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.870235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.871188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:17.871662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:17.873212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:17.873507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:17.874258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:17.874545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:17.874840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:17.884102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 56/119] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:17.885144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.899360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:17.904281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:17.905910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:17.906811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 64/120] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:17.908813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:17.997691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77216b32-54a9-48bc-92fc-f122e91eb809 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.011673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:18.035448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5561e-249d-4ac3-9333-68c08f73d4ed None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.038932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.039620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.039905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.040179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.041370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.043292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.050198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 57/121] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:18.050198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.071271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.071897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.072175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.072446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.073555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.075031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.076604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 65/122] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:18.077935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.136564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:18.167929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96a25b-2dd4-47cd-bb7e-34e862f76d90 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.181165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49ccbc-fcaf-466f-bcf5-cf44730e49bb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.205445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.206082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.206366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.206644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] PATH_INFO: `/v3/users/f1958564b3764d8a981784de403215d4` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.219787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.219939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] RBAC: Authorizing `identity:get_user(user_id=f1958564b3764d8a981784de403215d4)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.220410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.220680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.220952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.221301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.221614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:18.221921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.222980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 58/123] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:18 2022] GET /identity/v3/users/f1958564b3764d8a981784de403215d4 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:18.223392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.223757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.236999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 66/124] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:18.236999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.271536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:18.333106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac59111-3666-4dda-a531-e242ed23a131 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.341668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732f2ed-b2cd-459c-9501-de8e0ffd4153 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.360817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:18.367473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.368102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.368414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.368664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.378519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:18.379574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.380393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.380736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.381073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/796353704e344990a572797eb8638328/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:18.383294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:18.384325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:18.385296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:18.386101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53f8195f07e4713b63a46a6129a59d1, user_id=796353704e344990a572797eb8638328, role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.386182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 59/125] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:18.386182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.386318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.482984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 67/126] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:18 2022] PUT /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/796353704e344990a572797eb8638328/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:18.484106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.500673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:18.504263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1734-28b6-4bb4-b447-63a9a0bbc3fb None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.544023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.544726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.545443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.545932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.547017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.548699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.616365 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dc50e-9895-485f-a7ac-07f843a756b7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.653663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 60/127] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:11:18 2022] POST /identity/v3/projects => generated 302 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:18.654675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.663832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.664773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.665281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.665789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.776971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:18.782874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4c1ff-d362-403f-aaa3-0e782a2095cf None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:18.819125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.821141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.829483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:18.830334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:18.830785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:18.831196 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:18.832523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:18.834200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:18.836440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 61/128] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:18 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:18.837447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:18.869155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:19.021949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 68/129] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:19.023699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:19.025925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80ef21-f13a-49e0-bc6a-9468b6cc3e60 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:19.108140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:19.114729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.114729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.114729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.114729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:19.114729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:19.122734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:19.130674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:19.150684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 62/130] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:19.150684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:19.154624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5779d-cbfd-4626-9db4-9f87cea6c58f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:19.226884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:19.227528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.227812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.228088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] PATH_INFO: `/v3/projects/934753b97e9047f89a4b78277f71af05` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.230276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] RBAC: Authorizing `identity:get_project(project_id=934753b97e9047f89a4b78277f71af05)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:19.231773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:19.235394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 69/131] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:19 2022] GET /identity/v3/projects/934753b97e9047f89a4b78277f71af05 => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:19.239840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:19.290465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:19.313163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba89fd7-b5a2-418a-b22f-f4eed70ff312 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:19.359195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:19.360193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.360677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.361217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.362600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:19.364370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:19.368892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 63/132] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:19 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:19.370903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:19.445898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e553919-ba88-4f4b-94ca-77592b248b7f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:19.495901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:19.497125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.497411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.497891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.501898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:19.503403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:19.506263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 70/133] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:19 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:19.511941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:19.540578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:19.575453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31f147-0217-4ed1-924d-073f79cf9402 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:19.641392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acc1e5-58ea-4563-9c51-430b11aaf3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:19.642272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc1e5-58ea-4563-9c51-430b11aaf3cf None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.642563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc1e5-58ea-4563-9c51-430b11aaf3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.642868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc1e5-58ea-4563-9c51-430b11aaf3cf None admin] PATH_INFO: `/v3/projects/934753b97e9047f89a4b78277f71af05/users/796353704e344990a572797eb8638328/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.644912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acc1e5-58ea-4563-9c51-430b11aaf3cf None admin] RBAC: Authorizing `identity:create_grant(project_id=934753b97e9047f89a4b78277f71af05, user_id=796353704e344990a572797eb8638328, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:19.646562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acc1e5-58ea-4563-9c51-430b11aaf3cf None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:19.665806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c287-6a29-4b19-950d-1b000c1fda41 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:19.704624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:19.705257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.705535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.705809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.706534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] RBAC: Adding query filter params (role.id=2981ea0fce1c45c8b15e4bac0a72dd2f, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=f1958564b3764d8a981784de403215d4) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:19.706865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:19.708364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:19.714682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 71/134] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:19 2022] GET /identity/v3/role_assignments?user.id=f1958564b3764d8a981784de403215d4&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=2981ea0fce1c45c8b15e4bac0a72dd2f => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:19.715541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:19.764523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 64/135] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:19 2022] PUT /identity/v3/projects/934753b97e9047f89a4b78277f71af05/users/796353704e344990a572797eb8638328/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:19.779385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a8ba9-d385-4ac1-bbfb-32dbf6b90835 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.779693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a8ba9-d385-4ac1-bbfb-32dbf6b90835 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.780498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a8ba9-d385-4ac1-bbfb-32dbf6b90835 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.782006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 65/136] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:19.792137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d762e8-786c-46cf-88c5-d78a38ab367e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.792456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d762e8-786c-46cf-88c5-d78a38ab367e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.792810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d762e8-786c-46cf-88c5-d78a38ab367e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.807467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:19.824480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-89d762e8-786c-46cf-88c5-d78a38ab367e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:19.825766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-89d762e8-786c-46cf-88c5-d78a38ab367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:19.832808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf9ed7-ef95-43c0-a5d9-b3381eeac3f3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:19.836552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:19.873535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:19.874230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:19.875103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:19.875385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:19.876693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:19.878165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:19.880560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 72/137] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:19 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:19.888014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:20.012065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c236af-19eb-4d27-ab28-856595184e44 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:20.017079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 66/138] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:20.051679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:20.052331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:20.052614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:20.052889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/796353704e344990a572797eb8638328/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:20.054863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53f8195f07e4713b63a46a6129a59d1, user_id=796353704e344990a572797eb8638328, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:20.056355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:20.141844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 73/139] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:19 2022] PUT /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/796353704e344990a572797eb8638328/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:20.445525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d762e8-786c-46cf-88c5-d78a38ab367e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:20.572633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d762e8-786c-46cf-88c5-d78a38ab367e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:20.618151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e811b-d25a-462d-a6f2-b2499758525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:20.618831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e811b-d25a-462d-a6f2-b2499758525e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:20.619114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e811b-d25a-462d-a6f2-b2499758525e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:20.619389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e811b-d25a-462d-a6f2-b2499758525e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:20.627631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:20.635585 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e811b-d25a-462d-a6f2-b2499758525e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-771e811b-d25a-462d-a6f2-b2499758525e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:20.640421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:20.642508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:20.649019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 67/140] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:20.649640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:20.655536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:20.669476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48346f-ca19-4e7a-a82d-278ea78a1650 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:20.669769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48346f-ca19-4e7a-a82d-278ea78a1650 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:20.670058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48346f-ca19-4e7a-a82d-278ea78a1650 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:20.674892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 68/141] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:20.682935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45296b8f-d32f-4264-81db-62494675bbd3 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:20.682935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45296b8f-d32f-4264-81db-62494675bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:20.682935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45296b8f-d32f-4264-81db-62494675bbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:20.705472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-45296b8f-d32f-4264-81db-62494675bbd3 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:20.707564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-45296b8f-d32f-4264-81db-62494675bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:20.808398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50e1fd-fa2c-41d7-b68c-0c2ec518d085 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:20.828037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 69/142] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:20.892266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:20.892892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:20.893177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:20.893455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:20.894133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:20.894440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:20.895852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:20.907434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 74/143] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:20.913170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45296b8f-d32f-4264-81db-62494675bbd3 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:21.044367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45296b8f-d32f-4264-81db-62494675bbd3 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:21.074170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:21.084310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:21.085189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.085622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.089391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] PATH_INFO: `/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.089391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] RBAC: Authorizing `identity:get_user(user_id=ad9be8700f45469d8fb5f44fbb2eeb2c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:21.089391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:21.090988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 70/144] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:20 2022] GET /identity/v3/users/ad9be8700f45469d8fb5f44fbb2eeb2c => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:21.096487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:21.157775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:21.211061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:21.269687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f34cb7-3e3e-4f27-99cd-55a9e26fac31 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:21.337087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:21.337622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.337908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.338212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.366631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3f44d-54b0-440f-afd0-2d91383c70ac None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:21.444907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:21.445976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.446491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.446561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.464378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:21.470165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:21.471037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:21.471799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 71/145] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:11:21.478856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:21.519491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:21.520988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:21.577409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:21.615488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:21.646376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cc5a9-eb23-4d61-9493-b5b8f6f31436 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:21.652453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 75/146] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:21.659695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:21.711391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:21.712026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.712312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.712593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.713284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:21.713695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:21.714993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:21.734702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 72/147] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:21.742147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:21.874978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9866f-8b7d-44a0-8960-fff574e601a7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:21.892627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e90131-ba44-453c-8cfd-316f1e24ba98 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:21.912972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:21.936820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:21.937455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.938097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.938372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.940011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b901e-c130-4123-beec-282de57c9b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:21.940635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b901e-c130-4123-beec-282de57c9b94 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.940907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b901e-c130-4123-beec-282de57c9b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.941175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b901e-c130-4123-beec-282de57c9b94 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.942299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:21.943561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b901e-c130-4123-beec-282de57c9b94 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:21.944985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b901e-c130-4123-beec-282de57c9b94 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:21.946800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 76/148] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:21 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:21.947762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883384-cc62-4146-9a64-638c8b7817cb None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.948038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883384-cc62-4146-9a64-638c8b7817cb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.948310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883384-cc62-4146-9a64-638c8b7817cb None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.949614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 77/149] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:21.950471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:21.957504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 73/150] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:21 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:21.958757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efd6e-5c9c-4415-81cd-fa887d6bf5e9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:21.959042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efd6e-5c9c-4415-81cd-fa887d6bf5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:21.959318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efd6e-5c9c-4415-81cd-fa887d6bf5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:21.962669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:22.011624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a91efd6e-5c9c-4415-81cd-fa887d6bf5e9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:22.012836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a91efd6e-5c9c-4415-81cd-fa887d6bf5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:22.076575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:22.106115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5ce2a-f2fa-4598-9f0e-a48ba58ff834 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:22.147802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 78/151] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:22.149053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91efd6e-5c9c-4415-81cd-fa887d6bf5e9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:22.179646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87c1e2-c35d-41ef-b623-ed4dd5f27f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:22.180672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c1e2-c35d-41ef-b623-ed4dd5f27f05 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:22.181152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c1e2-c35d-41ef-b623-ed4dd5f27f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:22.181646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87c1e2-c35d-41ef-b623-ed4dd5f27f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:22.184085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87c1e2-c35d-41ef-b623-ed4dd5f27f05 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=ad9be8700f45469d8fb5f44fbb2eeb2c) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:22.184603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87c1e2-c35d-41ef-b623-ed4dd5f27f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:22.186395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87c1e2-c35d-41ef-b623-ed4dd5f27f05 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:22.199844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 74/152] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:21 2022] GET /identity/v3/role_assignments?user.id=ad9be8700f45469d8fb5f44fbb2eeb2c&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:22.288065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91efd6e-5c9c-4415-81cd-fa887d6bf5e9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:22.333109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf45b-512a-449c-bc1b-0c4f6a25c48a None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:22.333728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf45b-512a-449c-bc1b-0c4f6a25c48a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:22.334205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf45b-512a-449c-bc1b-0c4f6a25c48a None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:22.336221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 75/153] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:22.340012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:22.341584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a411f4-d474-4ea3-b4b3-ea45f7dfa0f8 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:22.342084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a411f4-d474-4ea3-b4b3-ea45f7dfa0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:22.342686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a411f4-d474-4ea3-b4b3-ea45f7dfa0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:22.348072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:22.350107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:22.351489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:22.371215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-06a411f4-d474-4ea3-b4b3-ea45f7dfa0f8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:22.375772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-06a411f4-d474-4ea3-b4b3-ea45f7dfa0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:22.429060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:22.475402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:22.487820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:22.489309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:22.507725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 76/154] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:22.510526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a411f4-d474-4ea3-b4b3-ea45f7dfa0f8 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:22.615569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a411f4-d474-4ea3-b4b3-ea45f7dfa0f8 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:22.645648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 79/155] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:22.650566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:22.651287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:22.651757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:22.652218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:22.652441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:22.653712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:22.655399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:22.657168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 77/156] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:22.662526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:22.727107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:22.745537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:22.765051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c63ff-e6fa-4fda-bc18-c2216826c8cf None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:22.794697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523109d-f769-47a8-830a-064fae5a7525 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:22.802590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:22.803236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:22.803491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:22.803919 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:22.804720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:22.806293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:22.833962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:22.834595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:22.834907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:22.835180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:22.836314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:22.837828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:22.839566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 80/157] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:22 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:22.846064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:22.921307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 78/158] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:11:22 2022] POST /identity/v3/users => generated 283 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:11:22.922961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:22.964943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4cf37-c799-4071-8291-7e3321d52ca5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:23.012032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:23.020709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb481f-1e5a-4ac3-ab94-8d8c1703db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:23.025613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb481f-1e5a-4ac3-ab94-8d8c1703db5b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.025613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb481f-1e5a-4ac3-ab94-8d8c1703db5b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.025613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb481f-1e5a-4ac3-ab94-8d8c1703db5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.025613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb481f-1e5a-4ac3-ab94-8d8c1703db5b None admin] RBAC: Adding query filter params (role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=796353704e344990a572797eb8638328) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:23.025613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb481f-1e5a-4ac3-ab94-8d8c1703db5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:23.028134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb481f-1e5a-4ac3-ab94-8d8c1703db5b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:23.036105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 81/159] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:22 2022] GET /identity/v3/role_assignments?user.id=796353704e344990a572797eb8638328&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:23.038238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a013b3-d6b5-4e90-b1b0-3d38dc9442c6 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.038685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a013b3-d6b5-4e90-b1b0-3d38dc9442c6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.039082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a013b3-d6b5-4e90-b1b0-3d38dc9442c6 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.040713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 82/160] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:23.048798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047aed7-ce9e-46d5-8e20-e0e80051e251 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.049403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047aed7-ce9e-46d5-8e20-e0e80051e251 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.049847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047aed7-ce9e-46d5-8e20-e0e80051e251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.085924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9047aed7-ce9e-46d5-8e20-e0e80051e251 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:23.086143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6f9cf-277e-4c1b-a6b0-2aeed92ddfc4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:23.087905 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9047aed7-ce9e-46d5-8e20-e0e80051e251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:23.096401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:23.137935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7ab31-7c80-4822-8456-b1f0d375f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:23.138598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7ab31-7c80-4822-8456-b1f0d375f7fa None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.138921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7ab31-7c80-4822-8456-b1f0d375f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.139732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7ab31-7c80-4822-8456-b1f0d375f7fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.156444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7ab31-7c80-4822-8456-b1f0d375f7fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-11b7ab31-7c80-4822-8456-b1f0d375f7fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:23.162596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:23.163610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:23.164394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:23.168740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 79/161] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:23.170212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac9ab-028c-428f-a826-c59387b0a0a5 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.170580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac9ab-028c-428f-a826-c59387b0a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.170926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac9ab-028c-428f-a826-c59387b0a0a5 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.172351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 80/162] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:23.173105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eac9ab-028c-428f-a826-c59387b0a0a5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:23.177690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 83/163] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:23.179889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c0796-7884-4375-8564-4aa237dbb103 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.179889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c0796-7884-4375-8564-4aa237dbb103 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.179889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c0796-7884-4375-8564-4aa237dbb103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.199507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e35c0796-7884-4375-8564-4aa237dbb103 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:23.200774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e35c0796-7884-4375-8564-4aa237dbb103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:23.293289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 84/164] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:23.304439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eac9ab-028c-428f-a826-c59387b0a0a5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:23.342592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:23.343557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.344003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.344467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.345313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:23.345781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:23.347673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:23.356534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 81/165] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:23.362408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c0796-7884-4375-8564-4aa237dbb103 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:23.446550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:23.506712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c0796-7884-4375-8564-4aa237dbb103 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:23.525815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:23.549875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:23.550419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.550750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.551064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] PATH_INFO: `/v3/users/f1958564b3764d8a981784de403215d4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.552161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] RBAC: Authorizing `identity:get_user(user_id=f1958564b3764d8a981784de403215d4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:23.553626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:23.557020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 85/166] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:23 2022] GET /identity/v3/users/f1958564b3764d8a981784de403215d4 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:23.563787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:23.596577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:23.741118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b1d4a-68f3-485e-a9c3-a57e8a5c1fb2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:23.760408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8144543-7279-4504-85bb-d6e5e262e116 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:23.783287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:23.783639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.783918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.784193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.794545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:23.796532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:23.810814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:23.812361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 82/167] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:23.813437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:23.820450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:23.821714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:23.821714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:23.822072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:23.825224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:23.826942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:23.830172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 86/168] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:23 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:23.831047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:23.936685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:23.953781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60553b96-6a5e-48cc-9866-6440193fd770 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:23.961988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474735b9-695e-4cab-89c5-6436167a549e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:24.062632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:24.063230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:24.063516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:24.063792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:24.069841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:24.070457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:24.070770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:24.071046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:24.076295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:24.076295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:24.076720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:24.084487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:24.090260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:24.091174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:24.092698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 83/169] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:24.093426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:24.093497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 87/170] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:24.098876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:24.171019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:24.245717 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bb329-51b2-460b-aa60-78acf0f6cb9d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:24.287792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d813b4-76f7-4ffc-90b5-2a6b8f40b906 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:24.293259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:24.293873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:24.294403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:24.294403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:24.295088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:24.295418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:24.301561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:24.316177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 88/171] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:24.317749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:24.348101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:24.348761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:24.349039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:24.349311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/f1958564b3764d8a981784de403215d4/roles/2981ea0fce1c45c8b15e4bac0a72dd2f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:24.351355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53f8195f07e4713b63a46a6129a59d1, user_id=f1958564b3764d8a981784de403215d4, role_id=2981ea0fce1c45c8b15e4bac0a72dd2f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:24.353354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:24.445677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 84/172] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:24 2022] PUT /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1/users/f1958564b3764d8a981784de403215d4/roles/2981ea0fce1c45c8b15e4bac0a72dd2f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:24.446311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:24.464353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709ff9-17e3-4bbf-aa9f-fc0f459d95e9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:24.475003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:24.487906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:24.508014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:24.508781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:24.509068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:24.509405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:24.510909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:24.512641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:24.514969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 89/173] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:24 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:24.521994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:24.687874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee63dda-23fd-4f99-83d6-0fb5d5c40bed None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:24.720994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c34666-a36e-43aa-a6e7-099aed51e597 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:24.743511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:24.744127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:24.744411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:24.744673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:24.784254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:24.784887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:24.785165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:24.785438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] PATH_INFO: `/v3/users/796353704e344990a572797eb8638328` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:24.786762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] RBAC: Authorizing `identity:get_user(user_id=796353704e344990a572797eb8638328)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:24.789340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:24.791845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 85/174] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:24 2022] GET /identity/v3/users/796353704e344990a572797eb8638328 => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:24.797991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:24.847329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:24.891812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:24.893347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:24.951324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:25.004725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db283cd1-18f4-42b4-a880-aaa6fab30d97 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:25.072428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:25.073045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:25.073321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:25.073588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:25.082013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 90/175] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:25.088515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:25.208049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:25.209441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:25.280482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:25.291044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:25.315769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 86/176] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:25.322569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010330e4-4814-45af-963e-6b3f0681d531 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:25.327998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:25.401381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:25.402458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:25.402771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:25.403052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:25.407528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:25.408988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:25.411045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 91/177] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:25 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:25.437248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:25.454740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bcde0-5b23-43f2-80c0-fcc3a0dc27f3 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:25.493771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:25.494664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:25.494949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:25.495222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] PATH_INFO: `/v3/projects/934753b97e9047f89a4b78277f71af05` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:25.496766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] RBAC: Authorizing `identity:get_project(project_id=934753b97e9047f89a4b78277f71af05)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:25.498382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:25.500265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 87/178] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:25 2022] GET /identity/v3/projects/934753b97e9047f89a4b78277f71af05 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:25.506001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:25.561742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:25.618370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d738d3c-0612-496d-8642-98eeffed4714 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:25.648000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a68c66-999b-4d0e-9986-c98c5952da0e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:25.654552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73d836-3512-4e99-b733-52c8a74ddc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:25.655731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d836-3512-4e99-b733-52c8a74ddc93 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:25.656194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d836-3512-4e99-b733-52c8a74ddc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:25.656464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d836-3512-4e99-b733-52c8a74ddc93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:25.657255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d836-3512-4e99-b733-52c8a74ddc93 None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=934753b97e9047f89a4b78277f71af05, user.id=796353704e344990a572797eb8638328) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:25.657563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d836-3512-4e99-b733-52c8a74ddc93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:25.659034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d836-3512-4e99-b733-52c8a74ddc93 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:25.665250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 88/179] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:25 2022] GET /identity/v3/role_assignments?user.id=796353704e344990a572797eb8638328&scope.project.id=934753b97e9047f89a4b78277f71af05&role.id=dbcd6da9a237441cb483965b19c21cff => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:25.719722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874637c-0952-44de-b10a-94a3deac667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:25.720366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874637c-0952-44de-b10a-94a3deac667f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:25.720643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874637c-0952-44de-b10a-94a3deac667f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:25.720918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874637c-0952-44de-b10a-94a3deac667f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:25.721689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874637c-0952-44de-b10a-94a3deac667f None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=796353704e344990a572797eb8638328) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:25.721999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874637c-0952-44de-b10a-94a3deac667f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:25.723531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874637c-0952-44de-b10a-94a3deac667f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:25.735178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 92/180] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:25 2022] GET /identity/v3/role_assignments?user.id=796353704e344990a572797eb8638328&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=dbcd6da9a237441cb483965b19c21cff => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:25.788602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d31e07-67bf-4e6f-b9a6-74f919a524c9 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:25.789100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d31e07-67bf-4e6f-b9a6-74f919a524c9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:25.789100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d31e07-67bf-4e6f-b9a6-74f919a524c9 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:25.790718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 89/181] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:25.796965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb4197-de6f-40c8-b0d0-8a7180f18c2f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:25.797249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb4197-de6f-40c8-b0d0-8a7180f18c2f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:25.797522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb4197-de6f-40c8-b0d0-8a7180f18c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:25.819780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e3bb4197-de6f-40c8-b0d0-8a7180f18c2f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:25.821010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e3bb4197-de6f-40c8-b0d0-8a7180f18c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:25.839605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:25.916984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 93/182] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:26.247316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985d95e-88c9-43ee-b7a0-dd6eeaa0985f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.247640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985d95e-88c9-43ee-b7a0-dd6eeaa0985f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.247852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985d95e-88c9-43ee-b7a0-dd6eeaa0985f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.249393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 90/183] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:26.250957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb4197-de6f-40c8-b0d0-8a7180f18c2f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:26.255460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ddc91-1e93-4977-a33c-24075b3df724 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.255738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ddc91-1e93-4977-a33c-24075b3df724 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.256005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ddc91-1e93-4977-a33c-24075b3df724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.282493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7b1ddc91-1e93-4977-a33c-24075b3df724 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:26.283729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7b1ddc91-1e93-4977-a33c-24075b3df724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:26.364640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 91/184] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:26.366161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951dd3-80ba-4689-b074-dde8dd2bcdb0 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.366443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951dd3-80ba-4689-b074-dde8dd2bcdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.366757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951dd3-80ba-4689-b074-dde8dd2bcdb0 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.367991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 92/185] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:26.376615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a89e7-fd29-4604-8b93-ab47f14654d4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.376891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a89e7-fd29-4604-8b93-ab47f14654d4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.377212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a89e7-fd29-4604-8b93-ab47f14654d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.382427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb4197-de6f-40c8-b0d0-8a7180f18c2f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:26.407503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-558a89e7-fd29-4604-8b93-ab47f14654d4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:26.409355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-558a89e7-fd29-4604-8b93-ab47f14654d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:26.422004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:26.423147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.423524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.423893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.437553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:26.444050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:26.445132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:26.446115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:26.448076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 94/186] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:26.450331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:26.454403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:26.464002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:26.494491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 93/187] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:26.558745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8cab8-fcfb-4929-abda-691fc7969c52 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:26.593249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:26.594197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.594676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.595136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.595975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:26.596451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:26.597998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:26.606371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 95/188] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:26.615030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a89e7-fd29-4604-8b93-ab47f14654d4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:26.653765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:26.723355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:26.725761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a89e7-fd29-4604-8b93-ab47f14654d4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:26.757795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958278ed-e9b0-49dc-92fa-f811026bf2d4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:26.763831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:26.765190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.765631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.766061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] PATH_INFO: `/v3/users/276691f052134ffe846baca45fb4598d` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.781473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] RBAC: Authorizing `identity:get_user(user_id=276691f052134ffe846baca45fb4598d)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:26.783579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:26.785648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 94/189] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:26 2022] GET /identity/v3/users/276691f052134ffe846baca45fb4598d => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:26.790690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:26.796060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:26.796677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.796977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.797246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.798340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:26.799930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:26.801467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 96/190] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:26.802148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:26.823689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:26.899052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75643a8-762e-4515-9f40-aa71ea2e9deb None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:26.911269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb97cd-8787-458c-ba0b-9b4918b464ff None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:26.934536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:26.935189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.935463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.935735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.946121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:26.948581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:26.949349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:26.949772 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:26.950189 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:26.950877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:26.952459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 95/191] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:26.953471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:26.960603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:27.036162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:27.037938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:27.044734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:27.062921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608d769-ed18-477b-9163-1705b0f86667 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:27.098602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd267c-73e4-474f-bdc0-0eb09d4bca5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:27.099163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd267c-73e4-474f-bdc0-0eb09d4bca5e None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.099438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd267c-73e4-474f-bdc0-0eb09d4bca5e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.099705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd267c-73e4-474f-bdc0-0eb09d4bca5e None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.100338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd267c-73e4-474f-bdc0-0eb09d4bca5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:27.101768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd267c-73e4-474f-bdc0-0eb09d4bca5e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:27.140784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 97/192] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:27.141623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:27.174666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:27.214036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 96/193] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:11:26 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:27.215398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c54c35-ee28-4799-91a4-7bdcb6ccdfed None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.215741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c54c35-ee28-4799-91a4-7bdcb6ccdfed None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.216280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c54c35-ee28-4799-91a4-7bdcb6ccdfed None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.219076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 97/194] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:27.219076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c54c35-ee28-4799-91a4-7bdcb6ccdfed None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:27.245413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c3633-a57c-4cfe-995e-54dde55ee9a7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:27.283896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda36e93-c29c-4ac7-a09e-6a0c2b6a67ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:27.284509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda36e93-c29c-4ac7-a09e-6a0c2b6a67ba None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.284785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda36e93-c29c-4ac7-a09e-6a0c2b6a67ba None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.285056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda36e93-c29c-4ac7-a09e-6a0c2b6a67ba None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.285734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda36e93-c29c-4ac7-a09e-6a0c2b6a67ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:27.286020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda36e93-c29c-4ac7-a09e-6a0c2b6a67ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:27.287486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda36e93-c29c-4ac7-a09e-6a0c2b6a67ba None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:27.298706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 98/195] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:27.298706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87499ac1-10f3-43bd-b3d1-9b6e8171cd4a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.298706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87499ac1-10f3-43bd-b3d1-9b6e8171cd4a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.298706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87499ac1-10f3-43bd-b3d1-9b6e8171cd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.316872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-87499ac1-10f3-43bd-b3d1-9b6e8171cd4a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:27.318077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-87499ac1-10f3-43bd-b3d1-9b6e8171cd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:27.351510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c54c35-ee28-4799-91a4-7bdcb6ccdfed None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:27.389519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:27.390144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.390417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.390770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.391892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:27.394380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:27.395280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 98/196] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:27 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:27.396047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:27.400162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 99/197] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:27.401521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87499ac1-10f3-43bd-b3d1-9b6e8171cd4a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:27.411253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:27.434023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:27.534030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87499ac1-10f3-43bd-b3d1-9b6e8171cd4a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:27.544267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5ae6c-ef09-49ab-966f-37fb503df83c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:27.581833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:27.582453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.582771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.583047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.583789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=276691f052134ffe846baca45fb4598d) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:27.584096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:27.585558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:27.590046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:27.590792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.591155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.591437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] PATH_INFO: `/v3/users/276691f052134ffe846baca45fb4598d` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.593249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=276691f052134ffe846baca45fb4598d)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:27.594020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 100/198] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:27 2022] GET /identity/v3/role_assignments?user.id=276691f052134ffe846baca45fb4598d&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:27.595134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:27.596901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 99/199] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:27 2022] GET /identity/v3/users/276691f052134ffe846baca45fb4598d => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:27.602882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:27.698329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:27.723313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:27.737422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f0f4a-cdf1-4b10-ab3a-8787af1deaa4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:27.776876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:27.777497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.777778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.778053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.811566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:27.854057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433b3fe-e4b3-414c-9222-d914a0d7a2c5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:27.893068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:27.894069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:27.894552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:27.895053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:27.906164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:27.912414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:27.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:27.914142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:27.917656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 100/200] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:11:27.926622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:27.929308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:27.930944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:28.087838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:28.098809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638ccb-e899-4b0c-9845-6364b55c9fd5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:28.105147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 101/201] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:28.111801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:28.162607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:28.163361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:28.163647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:28.163919 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:28.164594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:28.164952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:28.166346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:28.183986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:28.192718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 101/202] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:28.198528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:28.225036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7550977-a64a-47a7-90a6-12aa01e2af20 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:28.262512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:28.263254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:28.263528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:28.264307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:28.265581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:28.267185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:28.268979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 102/203] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:28 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:28.274444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:28.367130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:28.382295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596eb7-9516-48fc-a0e8-beda27c6796f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:28.413922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941280e1-6ab4-4617-a987-6838a440657c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:28.457982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:28.458592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:28.458903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:28.459176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:28.459906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] RBAC: Adding query filter params (role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=276691f052134ffe846baca45fb4598d) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:28.460213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:28.461647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:28.463893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:28.464520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:28.464796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:28.465067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] PATH_INFO: `/v3/users/f1958564b3764d8a981784de403215d4` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:28.466231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] RBAC: Authorizing `identity:get_user(user_id=f1958564b3764d8a981784de403215d4)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:28.468241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:28.469903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 102/204] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:28 2022] GET /identity/v3/users/f1958564b3764d8a981784de403215d4 => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:28.479567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:28.481612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 103/205] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:28 2022] GET /identity/v3/role_assignments?user.id=276691f052134ffe846baca45fb4598d&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:28.594063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:28.637310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d32d8-0479-42e0-a674-ad1fc3d03e3e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:28.686292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:28.686985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:28.687268 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:28.687537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:28.790216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:28.791741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:28.868559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:28.906749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 103/206] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:28.912205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:29.026989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31672-44f1-48a2-91f7-d154ad37e289 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:29.066303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:29.083353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3fc01-ebd9-42d4-97a5-addc75e29fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:29.083997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3fc01-ebd9-42d4-97a5-addc75e29fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:29.084278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3fc01-ebd9-42d4-97a5-addc75e29fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:29.084548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3fc01-ebd9-42d4-97a5-addc75e29fa7 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:29.085773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3fc01-ebd9-42d4-97a5-addc75e29fa7 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:29.087239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3fc01-ebd9-42d4-97a5-addc75e29fa7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:29.089099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 104/207] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:28 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:29.094818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:29.223647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089c2f2-f200-426d-9dae-3eebd7286ee4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:29.265867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655606c-ecac-4a0d-8b10-eb397a1364cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:29.266466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655606c-ecac-4a0d-8b10-eb397a1364cf None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:29.266760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655606c-ecac-4a0d-8b10-eb397a1364cf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:29.267030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655606c-ecac-4a0d-8b10-eb397a1364cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:29.267749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655606c-ecac-4a0d-8b10-eb397a1364cf None admin] RBAC: Adding query filter params (role.id=2981ea0fce1c45c8b15e4bac0a72dd2f, scope.project.id=c53f8195f07e4713b63a46a6129a59d1, user.id=f1958564b3764d8a981784de403215d4) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:29.268042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655606c-ecac-4a0d-8b10-eb397a1364cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:29.269478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655606c-ecac-4a0d-8b10-eb397a1364cf None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:29.275485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 104/208] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:29 2022] GET /identity/v3/role_assignments?user.id=f1958564b3764d8a981784de403215d4&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=2981ea0fce1c45c8b15e4bac0a72dd2f => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:29.644496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059b2c0-3611-4297-bccf-7eed8e35ac9a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:29.645009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059b2c0-3611-4297-bccf-7eed8e35ac9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:29.645122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059b2c0-3611-4297-bccf-7eed8e35ac9a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:29.646519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 105/209] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:29.652043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8e4b-b7f8-4ed8-a2fa-8cf70ca3b011 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:29.652570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8e4b-b7f8-4ed8-a2fa-8cf70ca3b011 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:29.653063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8e4b-b7f8-4ed8-a2fa-8cf70ca3b011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:29.675842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5d9c8e4b-b7f8-4ed8-a2fa-8cf70ca3b011 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:29.677377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5d9c8e4b-b7f8-4ed8-a2fa-8cf70ca3b011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:29.709379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:29.755875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 105/210] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:30.033897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059b2c0-3611-4297-bccf-7eed8e35ac9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:30.050910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0898b-2121-495f-952c-9c4229207f7c None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.051707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0898b-2121-495f-952c-9c4229207f7c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.051707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0898b-2121-495f-952c-9c4229207f7c None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.053158 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 106/211] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:30.056386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399b82c-63f8-470b-9611-162e4cc6059d None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.056866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399b82c-63f8-470b-9611-162e4cc6059d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.057252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399b82c-63f8-470b-9611-162e4cc6059d None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.058900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 107/212] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:30.059933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6ff0-afbf-49c6-8cfe-b12682c41097 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.060214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6ff0-afbf-49c6-8cfe-b12682c41097 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.060484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6ff0-afbf-49c6-8cfe-b12682c41097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.081178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-624b6ff0-afbf-49c6-8cfe-b12682c41097 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:30.082359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-624b6ff0-afbf-49c6-8cfe-b12682c41097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:30.148444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059b2c0-3611-4297-bccf-7eed8e35ac9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:30.153284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 108/213] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:30.155327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1c07b-4ab1-4774-ae5f-426f96f2e13f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.155834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1c07b-4ab1-4774-ae5f-426f96f2e13f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.156417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1c07b-4ab1-4774-ae5f-426f96f2e13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.179901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5fa1c07b-4ab1-4774-ae5f-426f96f2e13f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:30.181200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5fa1c07b-4ab1-4774-ae5f-426f96f2e13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:30.194453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:30.197670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.198138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.198586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.200736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:30.203017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:30.205614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 106/214] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:30.210469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:30.273815 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:30.279181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 109/215] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:30.319141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e694aff-a9b9-46b9-bc41-eb22c2309db8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:30.355889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96817c2-8def-4b6b-88cd-06ddaa0649cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:30.356492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96817c2-8def-4b6b-88cd-06ddaa0649cb None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.356767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96817c2-8def-4b6b-88cd-06ddaa0649cb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.357048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96817c2-8def-4b6b-88cd-06ddaa0649cb None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.358499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96817c2-8def-4b6b-88cd-06ddaa0649cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:30.359963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96817c2-8def-4b6b-88cd-06ddaa0649cb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:30.425667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1c07b-4ab1-4774-ae5f-426f96f2e13f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:30.452723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:30.466204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 107/216] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:11:30 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:11:30.551049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfcc0-1d79-4474-b272-9717866a0d21 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.551320 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfcc0-1d79-4474-b272-9717866a0d21 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.551596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dfcc0-1d79-4474-b272-9717866a0d21 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.552972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 108/217] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:30.556987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1c07b-4ab1-4774-ae5f-426f96f2e13f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:30.558638 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aac67-1c94-425e-ac0e-219b2304afbb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.559197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aac67-1c94-425e-ac0e-219b2304afbb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.559697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aac67-1c94-425e-ac0e-219b2304afbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.579417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6d7aac67-1c94-425e-ac0e-219b2304afbb None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:30.580952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6d7aac67-1c94-425e-ac0e-219b2304afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:30.596226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:30.596869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.597142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.597416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.610273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:30.614915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:30.615759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 110/218] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:30.616541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:30.664300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 109/219] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:30.665159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aac67-1c94-425e-ac0e-219b2304afbb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:30.715145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:30.725945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:30.744575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747e829-3a6a-4388-8989-ed61b789e05b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:30.800082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:30.800959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.801397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.801713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aac67-1c94-425e-ac0e-219b2304afbb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:30.801991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.824826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:30.829718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:30.830583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:30.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 111/220] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:30.837078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:30.850070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:30.851117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:30.851865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:30.852311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:30.853133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:30.853589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:30.855431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:30.865906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 110/221] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:30.872218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:30.989582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f3ad0-6a92-4779-809d-656570cf3406 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.025367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c88877-651a-4a8f-b58b-8fd46148e092 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.035018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.035635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.035919 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.036188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.037314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.038036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:31.038344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.040097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.050286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 112/222] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:31.055011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.061620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.074218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.074895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.078630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.078939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] PATH_INFO: `/v3/users/276691f052134ffe846baca45fb4598d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.080297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] RBAC: Authorizing `identity:get_user(user_id=276691f052134ffe846baca45fb4598d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.082853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.090151 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 111/223] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:30 2022] GET /identity/v3/users/276691f052134ffe846baca45fb4598d => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:31.094186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.193555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21b7a9-3323-4521-a499-6ede5c6a035e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.206860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9314c-5583-4c9e-9e67-c005e91d92eb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.240656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.241905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.242509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.243161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.244268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.244903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.245185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.245461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] PATH_INFO: `/v3/users/455ec42636ed47f5925c37bafffb3ae3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.257165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:31.261418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] RBAC: Authorizing `identity:get_user(user_id=455ec42636ed47f5925c37bafffb3ae3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.262896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:31.264680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:31.265845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:31.267084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:31.267689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 112/224] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/users/455ec42636ed47f5925c37bafffb3ae3 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:31.267689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.269503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 113/225] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:31.271232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.315596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.363016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.387491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c406f4-0f5e-4f05-9fb9-8d96cb499d12 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.406855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d88b7-4d51-4ab6-a722-9b4356a7e7e6 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.435000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.435725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.436014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.436496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.437408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.438896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.445097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 113/226] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:31.445097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.449971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.450771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.451718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.452154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.521707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.523426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.551294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f105261-4677-482f-aae6-d17d0cf78b74 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.552941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.587587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.588499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.588970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.589346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.589781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.590811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:31.591220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.592800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.601374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 114/227] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:31.606961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.632315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 114/228] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:31.633625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.711283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7602-b8d6-42e4-934e-bd90a3d2c908 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.739890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cf0b2-2e51-43ee-bbbc-b79f33a9ee6d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.745947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.746613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.746949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.747226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f/users/276691f052134ffe846baca45fb4598d/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.749707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c53322174e446daa46641a7229671f, user_id=276691f052134ffe846baca45fb4598d, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.751312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.780018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.780792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.781163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.781507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] PATH_INFO: `/v3/users/276691f052134ffe846baca45fb4598d` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:31.782736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] RBAC: Authorizing `identity:get_user(user_id=276691f052134ffe846baca45fb4598d)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:31.784248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:31.790602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 115/229] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/users/276691f052134ffe846baca45fb4598d => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:31.790602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.821637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 115/230] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:31 2022] PUT /identity/v3/projects/a0c53322174e446daa46641a7229671f/users/276691f052134ffe846baca45fb4598d/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:31.822937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:31.830641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.865451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:31.953659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694de6-66b2-45c7-9b39-fda5c02c5511 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:31.996313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:31.997939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:31.999209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:31.999719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.001164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:32.003307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:32.005276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 116/231] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:32.011663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:32.016947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cd744-b20d-427d-acf6-fde9a4c00844 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:32.063552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:32.063552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:32.064248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:32.064248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.065607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:32.067224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:32.069294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 116/232] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:31 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:32.074796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:32.172576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:32.187416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3732013-688a-4743-a7f3-859d50e109ef None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:32.193999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:32.197488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750636b5-d545-40a3-97d9-84c04d194a26 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:32.234969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bf470-af8f-4fa2-b009-b2ba0863d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:32.235637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bf470-af8f-4fa2-b009-b2ba0863d110 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:32.235919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bf470-af8f-4fa2-b009-b2ba0863d110 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:32.236194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bf470-af8f-4fa2-b009-b2ba0863d110 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f/users/276691f052134ffe846baca45fb4598d/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.239050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64036bd4-a4e9-4593-a32f-73f47c9b670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:32.239656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64036bd4-a4e9-4593-a32f-73f47c9b670b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:32.239759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bf470-af8f-4fa2-b009-b2ba0863d110 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c53322174e446daa46641a7229671f, user_id=276691f052134ffe846baca45fb4598d, role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:32.240008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64036bd4-a4e9-4593-a32f-73f47c9b670b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:32.240248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64036bd4-a4e9-4593-a32f-73f47c9b670b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.240984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64036bd4-a4e9-4593-a32f-73f47c9b670b None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=455ec42636ed47f5925c37bafffb3ae3) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:32.241295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64036bd4-a4e9-4593-a32f-73f47c9b670b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:32.242367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3bf470-af8f-4fa2-b009-b2ba0863d110 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:32.242749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64036bd4-a4e9-4593-a32f-73f47c9b670b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:32.250041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 117/233] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:32 2022] GET /identity/v3/role_assignments?user.id=455ec42636ed47f5925c37bafffb3ae3&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=dbcd6da9a237441cb483965b19c21cff => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:32.385033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 117/234] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:32 2022] PUT /identity/v3/projects/a0c53322174e446daa46641a7229671f/users/276691f052134ffe846baca45fb4598d/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:32.793279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d437486-bb13-4375-90e3-db5d85c3c5f3 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:32.793689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d437486-bb13-4375-90e3-db5d85c3c5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:32.793865 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d437486-bb13-4375-90e3-db5d85c3c5f3 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.795513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 118/235] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:32.800715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb99cd-a79f-492a-a1f6-83ab1834e2a0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:32.801153 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb99cd-a79f-492a-a1f6-83ab1834e2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:32.801251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb99cd-a79f-492a-a1f6-83ab1834e2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.817594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e2095-a11c-4230-9037-dac029850f14 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:32.817888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e2095-a11c-4230-9037-dac029850f14 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:32.818146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e2095-a11c-4230-9037-dac029850f14 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.819520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 119/236] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:32.823336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8ebb99cd-a79f-492a-a1f6-83ab1834e2a0 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:32.824541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8ebb99cd-a79f-492a-a1f6-83ab1834e2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:32.831232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52470c-5827-4691-9a41-120204e6499d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:32.831514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52470c-5827-4691-9a41-120204e6499d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:32.831794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52470c-5827-4691-9a41-120204e6499d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:32.863179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7c52470c-5827-4691-9a41-120204e6499d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:32.867153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7c52470c-5827-4691-9a41-120204e6499d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:32.915890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 118/237] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:32.967794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 120/238] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:33.244463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb99cd-a79f-492a-a1f6-83ab1834e2a0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:33.336250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:33.354380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb99cd-a79f-492a-a1f6-83ab1834e2a0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:33.371308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52470c-5827-4691-9a41-120204e6499d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:33.417831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:33.418466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:33.418775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:33.420863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:33.423200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:33.424653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:33.426294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 119/239] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:33.433676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:33.561191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:33.578516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52470c-5827-4691-9a41-120204e6499d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:33.581408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e364d4-6c56-4ba3-9f51-65268f2b0809 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:33.625781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:33.626495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:33.626818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:33.627092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:33.627752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:33.629211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:33.630705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:33.631341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:33.631620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:33.631890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:33.643697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:33.649236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:33.650186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:33.651030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 121/240] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:11:33.658180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:33.729232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 120/241] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:11:33 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:33.778791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b74084-12d0-4554-b5b9-bf25a45fd30f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:33.829706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:33.830290 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:33.830571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:33.830974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:33.831677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:33.831982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:33.833409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:33.847946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 122/242] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:33.853384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:33.864699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:33.977385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e1815-7269-4430-9d58-5e14afc6ac9a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:34.015027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05d991-d131-4a73-8467-19b9f4b43863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:34.016056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05d991-d131-4a73-8467-19b9f4b43863 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.016337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05d991-d131-4a73-8467-19b9f4b43863 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.016668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05d991-d131-4a73-8467-19b9f4b43863 None admin] PATH_INFO: `/v3/users/4be25d53479248e4948b98b9a2fa0245` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.031095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05d991-d131-4a73-8467-19b9f4b43863 None admin] RBAC: Authorizing `identity:get_user(user_id=4be25d53479248e4948b98b9a2fa0245)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:34.032570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05d991-d131-4a73-8467-19b9f4b43863 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:34.034310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 121/243] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:33 2022] GET /identity/v3/users/4be25d53479248e4948b98b9a2fa0245 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:34.040130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:34.059941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:34.134894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010209-db1b-4dcd-b19f-47035d6dc225 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.135075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010209-db1b-4dcd-b19f-47035d6dc225 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.135352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010209-db1b-4dcd-b19f-47035d6dc225 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.136900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 122/244] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:34.141779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc651908-9cd6-470a-b1bc-0a0d6cf7af6a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.142063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc651908-9cd6-470a-b1bc-0a0d6cf7af6a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.142343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc651908-9cd6-470a-b1bc-0a0d6cf7af6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.160550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8bb6d-dd82-4aa9-8893-9f83264c7bd0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:34.163717 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bc651908-9cd6-470a-b1bc-0a0d6cf7af6a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:34.164921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bc651908-9cd6-470a-b1bc-0a0d6cf7af6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:34.195914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:34.196894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.197477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.197948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.255672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:34.267740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 123/245] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:34.283665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:34.285661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:34.303897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:34.390637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 123/246] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:34.396980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc651908-9cd6-470a-b1bc-0a0d6cf7af6a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:34.505938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc651908-9cd6-470a-b1bc-0a0d6cf7af6a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:34.542737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:34.543370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.543652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.543927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.545085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:34.546525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:34.548531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 124/247] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:34 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:34.554021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:34.598272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:34.668949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:34.675804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a99054-5190-425f-b886-b94579bbe877 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:34.701938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:34.714257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe57dc5-41ea-4ab7-839e-d30d42f4daa4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:34.753319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe824a5-6fb0-4192-9d94-1ecb421ec03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:34.754327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe824a5-6fb0-4192-9d94-1ecb421ec03c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.754882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe824a5-6fb0-4192-9d94-1ecb421ec03c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.755465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe824a5-6fb0-4192-9d94-1ecb421ec03c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.756831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ca013-3845-440c-9d9e-ffd1ae25b2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:34.757444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca013-3845-440c-9d9e-ffd1ae25b2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.757722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca013-3845-440c-9d9e-ffd1ae25b2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.757994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca013-3845-440c-9d9e-ffd1ae25b2f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.758760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ca013-3845-440c-9d9e-ffd1ae25b2f7 None admin] RBAC: Adding query filter params (role.id=2981ea0fce1c45c8b15e4bac0a72dd2f, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=4be25d53479248e4948b98b9a2fa0245) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:34.759070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ca013-3845-440c-9d9e-ffd1ae25b2f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:34.760534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ca013-3845-440c-9d9e-ffd1ae25b2f7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:34.769001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe824a5-6fb0-4192-9d94-1ecb421ec03c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cbe824a5-6fb0-4192-9d94-1ecb421ec03c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:34.774285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:34.775216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:34.776124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 125/248] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:34.778304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9a51-0570-419e-be69-94d352a4956b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.778304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9a51-0570-419e-be69-94d352a4956b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.778304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9a51-0570-419e-be69-94d352a4956b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.779281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 126/250] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:34.779449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 124/250] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:34 2022] GET /identity/v3/role_assignments?user.id=4be25d53479248e4948b98b9a2fa0245&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=2981ea0fce1c45c8b15e4bac0a72dd2f => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:34.780308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257de700-d955-497b-8b9e-80cb118cf9e5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.780581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257de700-d955-497b-8b9e-80cb118cf9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.780857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257de700-d955-497b-8b9e-80cb118cf9e5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.782270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 127/251] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:34.785190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee589-0971-4559-ab34-76ca3800392a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.785491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257de700-d955-497b-8b9e-80cb118cf9e5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:34.785836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee589-0971-4559-ab34-76ca3800392a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.786284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee589-0971-4559-ab34-76ca3800392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.827804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cdaee589-0971-4559-ab34-76ca3800392a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:34.829394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cdaee589-0971-4559-ab34-76ca3800392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:34.918740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257de700-d955-497b-8b9e-80cb118cf9e5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:34.946833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:34.959497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:34.960239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.960521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.960901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.961618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:34.961932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:34.963363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:34.966129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 125/252] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:34.967908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55219006-4e0a-4c79-8a1b-76a5064dcf61 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:34.968422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55219006-4e0a-4c79-8a1b-76a5064dcf61 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:34.968908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55219006-4e0a-4c79-8a1b-76a5064dcf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:34.972537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 128/253] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:34.979164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:34.991138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-55219006-4e0a-4c79-8a1b-76a5064dcf61 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:34.992727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-55219006-4e0a-4c79-8a1b-76a5064dcf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:35.108373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 126/254] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:35.142959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c74a6-1461-4901-94d5-60cf361a3ad4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:35.191138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:35.191769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:35.192045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:35.192316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] PATH_INFO: `/v3/users/276691f052134ffe846baca45fb4598d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:35.193463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=276691f052134ffe846baca45fb4598d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:35.194952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:35.196947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 129/255] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:34 2022] GET /identity/v3/users/276691f052134ffe846baca45fb4598d => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:35.202116 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55219006-4e0a-4c79-8a1b-76a5064dcf61 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:35.279048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:35.354205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55219006-4e0a-4c79-8a1b-76a5064dcf61 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:35.371036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:35.402104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:35.406808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:35.406808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:35.406808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:35.491017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:35.538181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:35.539692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:35.561894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb05d4-6a8c-4dbe-90ac-23ab4806a2a0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:35.623024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:35.628254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:35.628893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:35.629171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:35.629618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:35.651954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:35.657980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:35.658965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:35.659757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:35.662418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 130/256] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:35.671036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:35.734882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 127/257] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:35.735568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:35.837241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa18cb-6e27-463a-94b2-9072a144e13f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:35.866989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:35.902035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:35.902675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:35.903003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:35.903554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:35.905024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:35.905473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:35.907022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d22df3-ac90-4f55-a57f-88b186be7ca6 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:35.907170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:35.918241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 131/258] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:35.919203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.022853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb431-9cfd-4a86-96ec-d56c09ac025f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.023854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.024492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.024768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.025039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.043032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:36.044078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:36.049082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:36.049932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:36.050756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:36.052979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 128/259] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:36.053958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.067556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.068190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.068621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.069039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.076045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.078433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.083263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 132/260] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:35 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:36.084386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.164407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ae37-69db-426a-9e12-818e99019d68 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.165912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:36.202493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.203667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.204034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] PATH_INFO: `/v3/users/455ec42636ed47f5925c37bafffb3ae3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.206132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] RBAC: Authorizing `identity:get_user(user_id=455ec42636ed47f5925c37bafffb3ae3)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.208439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.210376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 129/261] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:36 2022] GET /identity/v3/users/455ec42636ed47f5925c37bafffb3ae3 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:36.211311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.289077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455726-3894-465b-b5bf-5de751a2ebdb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.291030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:36.330670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c6c68-e751-436e-a1e3-21ab1b254646 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.369053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.370218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.370679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.371090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.371949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=276691f052134ffe846baca45fb4598d) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:36.372408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.374522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.381085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 130/262] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:36 2022] GET /identity/v3/role_assignments?user.id=276691f052134ffe846baca45fb4598d&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:36.382052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.399397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.400035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.400337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.400592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.401290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:36.401594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.403050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.428694 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 133/263] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:36.443748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.508794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650d61e-035e-454b-a7fd-6e3499265d89 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.546551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:36.549439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.550241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.550840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.551295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.552597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.562369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.562369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 131/264] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:36 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:36.562369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.641485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e143a-7047-4374-97fb-7ddaf9f479d4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.648119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:36.669449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b2de-aed0-4784-8b8c-232fbabc5778 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.701142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.701771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.702328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.702433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] PATH_INFO: `/v3/users/276691f052134ffe846baca45fb4598d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.707337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] RBAC: Authorizing `identity:get_user(user_id=276691f052134ffe846baca45fb4598d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.708298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.709590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837bd3a-7705-499b-87b8-7fafb1611875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.709941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 134/265] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:36 2022] GET /identity/v3/users/276691f052134ffe846baca45fb4598d => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:36.710628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837bd3a-7705-499b-87b8-7fafb1611875 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.711127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837bd3a-7705-499b-87b8-7fafb1611875 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.711567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837bd3a-7705-499b-87b8-7fafb1611875 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f/users/455ec42636ed47f5925c37bafffb3ae3/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.713693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837bd3a-7705-499b-87b8-7fafb1611875 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c53322174e446daa46641a7229671f, user_id=455ec42636ed47f5925c37bafffb3ae3, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.715415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837bd3a-7705-499b-87b8-7fafb1611875 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.715738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:36.817268 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 132/266] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:36 2022] PUT /identity/v3/projects/a0c53322174e446daa46641a7229671f/users/455ec42636ed47f5925c37bafffb3ae3/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:11:36.832895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920617bb-3cac-4e49-8ceb-6badd1d98978 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:36.878190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b496-9f44-4a0e-ab87-38fbe4abf6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:36.878936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b496-9f44-4a0e-ab87-38fbe4abf6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.879232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b496-9f44-4a0e-ab87-38fbe4abf6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.879504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b496-9f44-4a0e-ab87-38fbe4abf6e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.908349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9bb72-b061-494f-bafe-747f595eb52f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.908909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9bb72-b061-494f-bafe-747f595eb52f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.909354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9bb72-b061-494f-bafe-747f595eb52f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.915496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 133/267] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:36.933192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:36.935242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b1fa-4089-4f0e-84a8-d04a11632f31 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:36.935728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b1fa-4089-4f0e-84a8-d04a11632f31 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:36.936192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908b1fa-4089-4f0e-84a8-d04a11632f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:36.969507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b496-9f44-4a0e-ab87-38fbe4abf6e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:36.971231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b496-9f44-4a0e-ab87-38fbe4abf6e5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:36.977944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1908b1fa-4089-4f0e-84a8-d04a11632f31 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:36.977944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1908b1fa-4089-4f0e-84a8-d04a11632f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:37.076796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:37.086039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 134/268] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:37.128425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 135/269] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:37.137248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b1fa-4089-4f0e-84a8-d04a11632f31 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:37.247177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03f20-de16-4b0a-9a41-6dfb0f9a16db None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.247496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03f20-de16-4b0a-9a41-6dfb0f9a16db None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.248350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03f20-de16-4b0a-9a41-6dfb0f9a16db None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.249834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 136/270] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:37.251327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908b1fa-4089-4f0e-84a8-d04a11632f31 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:37.254769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c136a8-1ac2-419a-89bd-a05c8409d2eb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.255057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c136a8-1ac2-419a-89bd-a05c8409d2eb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.255337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c136a8-1ac2-419a-89bd-a05c8409d2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.273999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-99c136a8-1ac2-419a-89bd-a05c8409d2eb None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:37.275339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-99c136a8-1ac2-419a-89bd-a05c8409d2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:37.289089 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497b7fe-eaa7-4136-ac66-3a0e3fdcb734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:37.289768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497b7fe-eaa7-4136-ac66-3a0e3fdcb734 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.290062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497b7fe-eaa7-4136-ac66-3a0e3fdcb734 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.290334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497b7fe-eaa7-4136-ac66-3a0e3fdcb734 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.292101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497b7fe-eaa7-4136-ac66-3a0e3fdcb734 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:37.295699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497b7fe-eaa7-4136-ac66-3a0e3fdcb734 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:37.296079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 135/271] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:37 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:37.297656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55c81-5b9a-476e-9398-468a24263dfc None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.298262 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55c81-5b9a-476e-9398-468a24263dfc None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.298551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55c81-5b9a-476e-9398-468a24263dfc None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.300265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 136/272] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:37.303528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf55c81-5b9a-476e-9398-468a24263dfc None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:37.363052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 137/273] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:37.364319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df02e-c2af-4ebb-8e6c-7fb901ca0c3a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.364604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df02e-c2af-4ebb-8e6c-7fb901ca0c3a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.364879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018df02e-c2af-4ebb-8e6c-7fb901ca0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.369749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:37.370856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:37.384372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-018df02e-c2af-4ebb-8e6c-7fb901ca0c3a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:37.385635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-018df02e-c2af-4ebb-8e6c-7fb901ca0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:37.415166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf55c81-5b9a-476e-9398-468a24263dfc None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:37.456741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:37.457642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.457913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.458180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.459056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] RBAC: Adding query filter params (role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=276691f052134ffe846baca45fb4598d) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:37.459674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:37.462326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:37.469939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 137/274] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:37 2022] GET /identity/v3/role_assignments?user.id=276691f052134ffe846baca45fb4598d&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:37.470758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:37.474019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 138/275] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:37.589477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3559a2-86d0-472a-8f4d-9af1b69af78c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:37.617418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:37.634306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:37.635329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.635810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.636280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.647177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:37.652219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:37.659542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:37.660332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 138/276] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:37.662534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018df02e-c2af-4ebb-8e6c-7fb901ca0c3a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:37.774334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018df02e-c2af-4ebb-8e6c-7fb901ca0c3a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:37.783394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:37.823790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:37.829007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:37.829626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.829905 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.830177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.831101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:37.845941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:37.845941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:37.845941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 139/277] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:37.845941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:37.913113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a7dd1-3093-478f-921b-8ea42b31ff53 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:37.952901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:37.953529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:37.953804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:37.958063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:37.958063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:37.962688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44587e54-c74f-4bf9-a7c7-86e6454e87b5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:37.963127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:37.963127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 139/278] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:37.963127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:37.998747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.000171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.000459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.000737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.012100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:38.014426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:38.017834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:38.018727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:38.019641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 140/279] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:38.020292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:38.073769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20776fd4-fd12-461f-8fa0-a30b36829653 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:38.077362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:38.111682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.112235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.112516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.112790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] PATH_INFO: `/v3/users/31726794c42d4eb19f9ebedba8f8f35b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.126498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] RBAC: Authorizing `identity:get_user(user_id=31726794c42d4eb19f9ebedba8f8f35b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:38.127214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec640384-6aa6-4323-8cd9-2b8b06932450 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:38.127990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:38.130115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 140/280] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:37 2022] GET /identity/v3/users/31726794c42d4eb19f9ebedba8f8f35b => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:38.131548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:38.165097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.165713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.165998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.166353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.166940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:38.168381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:38.239706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bcb76-dc55-460b-8ddd-6b2987f08b6c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:38.249183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:38.272824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.273485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.273761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.274028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.274710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:38.275010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:38.276373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:38.284142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 141/281] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:38.284961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:38.293153 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 141/282] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:11:38 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:38.295757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:38.425858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008fc39-e566-4224-aa05-3810dfbf45bc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:38.458733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d146-bc98-4dc6-8d32-1a48473c4272 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:38.470733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:38.474342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.476482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.476482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.476482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.476826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, user.id=31726794c42d4eb19f9ebedba8f8f35b, scope.system=all) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:38.478680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:38.478995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:38.486146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 142/283] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 12:11:38 2022] GET /identity/v3/role_assignments?user.id=31726794c42d4eb19f9ebedba8f8f35b&scope.system=all&role.id=dbcd6da9a237441cb483965b19c21cff => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:38.509594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.510252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.510591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.510899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] PATH_INFO: `/v3/users/4be25d53479248e4948b98b9a2fa0245` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.512140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] RBAC: Authorizing `identity:get_user(user_id=4be25d53479248e4948b98b9a2fa0245)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:38.513855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:38.515990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 142/284] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:38 2022] GET /identity/v3/users/4be25d53479248e4948b98b9a2fa0245 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:38.521778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:38.563848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:38.639788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aa0fb-7b79-4f05-a752-dcb4bbf362c8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:38.676312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a726-d33f-4a49-92bd-d8af2359680b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.677187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a726-d33f-4a49-92bd-d8af2359680b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.677615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a726-d33f-4a49-92bd-d8af2359680b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.678050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a726-d33f-4a49-92bd-d8af2359680b None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.679356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a726-d33f-4a49-92bd-d8af2359680b None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:38.680973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a726-d33f-4a49-92bd-d8af2359680b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:38.683135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 143/285] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:38 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:38.689136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:38.798394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11ecbc-d39e-4538-bcee-588760ab9c10 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:38.835727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab681b-398f-4a4b-9481-cb9c2609cec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:38.836730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab681b-398f-4a4b-9481-cb9c2609cec9 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:38.837157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab681b-398f-4a4b-9481-cb9c2609cec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:38.837578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab681b-398f-4a4b-9481-cb9c2609cec9 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f/users/4be25d53479248e4948b98b9a2fa0245/roles/2981ea0fce1c45c8b15e4bac0a72dd2f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:38.839795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab681b-398f-4a4b-9481-cb9c2609cec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c53322174e446daa46641a7229671f, user_id=4be25d53479248e4948b98b9a2fa0245, role_id=2981ea0fce1c45c8b15e4bac0a72dd2f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:38.841451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab681b-398f-4a4b-9481-cb9c2609cec9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:38.923593 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 143/286] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:11:38 2022] PUT /identity/v3/projects/a0c53322174e446daa46641a7229671f/users/4be25d53479248e4948b98b9a2fa0245/roles/2981ea0fce1c45c8b15e4bac0a72dd2f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:39.038969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a6d60-ff31-47c3-8223-574018255069 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:39.039252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a6d60-ff31-47c3-8223-574018255069 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:39.039525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a6d60-ff31-47c3-8223-574018255069 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:39.041168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 144/287] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:39.047212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d27b4-3e2f-43e8-9e25-51b678aae427 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:39.047703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d27b4-3e2f-43e8-9e25-51b678aae427 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:39.047794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d27b4-3e2f-43e8-9e25-51b678aae427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:39.066167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a64d27b4-3e2f-43e8-9e25-51b678aae427 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:39.067407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a64d27b4-3e2f-43e8-9e25-51b678aae427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:39.114467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:39.163891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 144/288] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:39.437027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a6d60-ff31-47c3-8223-574018255069 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:39.565738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a6d60-ff31-47c3-8223-574018255069 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:39.567315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:39.601150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:39.601768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:39.602041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:39.602306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:39.617469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:39.622140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:39.623104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:39.623900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 145/289] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:39.629427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d27b4-3e2f-43e8-9e25-51b678aae427 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:39.738494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d27b4-3e2f-43e8-9e25-51b678aae427 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:39.775519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:39.776117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:39.776395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:39.776658 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:39.777321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:39.777614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:39.779135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:39.788786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 145/290] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:39.794972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:39.912920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa85ebc-dd19-460a-8985-90892b011a94 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:39.955296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:39.955912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:39.956185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:39.956453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] PATH_INFO: `/v3/users/455ec42636ed47f5925c37bafffb3ae3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:39.961024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] RBAC: Authorizing `identity:get_user(user_id=455ec42636ed47f5925c37bafffb3ae3)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:39.962577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:39.964194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 146/291] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:39 2022] GET /identity/v3/users/455ec42636ed47f5925c37bafffb3ae3 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:39.970383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:40.007187 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:40.080924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347cbdb-b767-4741-93d6-f37bff9fb183 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:40.116902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:40.117499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:40.117773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:40.118037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:40.186547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:40.188026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:40.230644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:40.327218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 146/292] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:40.332609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:40.389889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:40.491430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0874c-1975-484f-83d6-fe1c87bfafde None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:40.535902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e39b6-3c38-499c-961e-f6905463169b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:40.536530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e39b6-3c38-499c-961e-f6905463169b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:40.536806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e39b6-3c38-499c-961e-f6905463169b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:40.537073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e39b6-3c38-499c-961e-f6905463169b None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:40.538199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e39b6-3c38-499c-961e-f6905463169b None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:40.539672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e39b6-3c38-499c-961e-f6905463169b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:40.541462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 147/293] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:40 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:40.547243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:40.693244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2fbf7-6223-497f-9f4c-e7fbe902b8b9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:40.736278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:40.742775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6c72b-d84e-4d56-891f-32059663c6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:40.743383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6c72b-d84e-4d56-891f-32059663c6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:40.743659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6c72b-d84e-4d56-891f-32059663c6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:40.743932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6c72b-d84e-4d56-891f-32059663c6b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:40.744659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6c72b-d84e-4d56-891f-32059663c6b7 None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=455ec42636ed47f5925c37bafffb3ae3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:40.744966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6c72b-d84e-4d56-891f-32059663c6b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:40.746398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6c72b-d84e-4d56-891f-32059663c6b7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:40.755400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 147/294] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:40 2022] GET /identity/v3/role_assignments?user.id=455ec42636ed47f5925c37bafffb3ae3&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=dbcd6da9a237441cb483965b19c21cff => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:40.929738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d797746-9a67-4136-9c77-7f2f239cc9c8 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:40.930239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d797746-9a67-4136-9c77-7f2f239cc9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:40.930344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d797746-9a67-4136-9c77-7f2f239cc9c8 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:40.931928 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 148/295] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:40.934690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cab55-ceb6-44e7-84c9-f5d6d8b1b57e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:40.935000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cab55-ceb6-44e7-84c9-f5d6d8b1b57e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:40.935276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cab55-ceb6-44e7-84c9-f5d6d8b1b57e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:40.936700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 148/296] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:40.937178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93d465-f4f4-4dd9-96ca-1bd7ed6c768a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:40.937488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93d465-f4f4-4dd9-96ca-1bd7ed6c768a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:40.937807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93d465-f4f4-4dd9-96ca-1bd7ed6c768a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:40.941935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac905-89ee-40a8-aef9-de82707daabb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:40.942222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac905-89ee-40a8-aef9-de82707daabb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:40.942498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aac905-89ee-40a8-aef9-de82707daabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:40.961904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9b93d465-f4f4-4dd9-96ca-1bd7ed6c768a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:40.962185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b9aac905-89ee-40a8-aef9-de82707daabb None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:40.963045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9b93d465-f4f4-4dd9-96ca-1bd7ed6c768a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:40.963379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b9aac905-89ee-40a8-aef9-de82707daabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:41.038224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 149/297] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:41.038884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 149/298] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:41.321615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aac905-89ee-40a8-aef9-de82707daabb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:41.334167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6cb23-cbf4-49d2-b4b4-45c4352bebcb None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.334484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6cb23-cbf4-49d2-b4b4-45c4352bebcb None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.334734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6cb23-cbf4-49d2-b4b4-45c4352bebcb None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.336254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 150/299] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:41.342342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f32e1a-a69d-4fe2-8ae7-55de0406bbee None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.342912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f32e1a-a69d-4fe2-8ae7-55de0406bbee None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.343622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f32e1a-a69d-4fe2-8ae7-55de0406bbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.365133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-02f32e1a-a69d-4fe2-8ae7-55de0406bbee None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:41.366376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-02f32e1a-a69d-4fe2-8ae7-55de0406bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:41.389892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:41.436828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aac905-89ee-40a8-aef9-de82707daabb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:41.437273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 151/300] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:41.438111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f32e1a-a69d-4fe2-8ae7-55de0406bbee None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:41.461093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:41.473769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b66301-4d8e-443b-8fbe-e7f197bb6a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:41.475169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b66301-4d8e-443b-8fbe-e7f197bb6a18 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.475451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b66301-4d8e-443b-8fbe-e7f197bb6a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.475717 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b66301-4d8e-443b-8fbe-e7f197bb6a18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.488342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b66301-4d8e-443b-8fbe-e7f197bb6a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-35b66301-4d8e-443b-8fbe-e7f197bb6a18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:41.493063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:41.494120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:41.495118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:41.499267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 150/301] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:41.500684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb239c47-4346-4a2b-89e9-8d8b2cbc0909 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.500969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb239c47-4346-4a2b-89e9-8d8b2cbc0909 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.501244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb239c47-4346-4a2b-89e9-8d8b2cbc0909 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.502677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 151/302] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:41.504549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb239c47-4346-4a2b-89e9-8d8b2cbc0909 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:41.554040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f32e1a-a69d-4fe2-8ae7-55de0406bbee None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:41.590333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521b8d9-d826-471c-9a58-c5e6a05ef2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:41.591873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521b8d9-d826-471c-9a58-c5e6a05ef2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.592344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521b8d9-d826-471c-9a58-c5e6a05ef2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.592783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521b8d9-d826-471c-9a58-c5e6a05ef2d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.603753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521b8d9-d826-471c-9a58-c5e6a05ef2d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-6521b8d9-d826-471c-9a58-c5e6a05ef2d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:41.609031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:41.609933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:41.610762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 152/303] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:41.611444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741adf7f-c172-48ca-be77-671fa923f461 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.612068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741adf7f-c172-48ca-be77-671fa923f461 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.612068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741adf7f-c172-48ca-be77-671fa923f461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.615406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb239c47-4346-4a2b-89e9-8d8b2cbc0909 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:41.630150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-741adf7f-c172-48ca-be77-671fa923f461 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:41.631404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-741adf7f-c172-48ca-be77-671fa923f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:41.647309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:41.651031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:41.651631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.651908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.652175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.652828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:41.653125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:41.655161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:41.663569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 152/304] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:41.663569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:41.708082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 153/305] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:41.708926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741adf7f-c172-48ca-be77-671fa923f461 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:41.718066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:41.775597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2bc73-da57-40ca-98f3-5e909b5642b2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:41.813222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:41.813824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.814103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.814433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.815058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:41.815359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:41.816720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:41.819501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741adf7f-c172-48ca-be77-671fa923f461 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:41.826199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 153/306] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:41.831250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:41.855323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:41.855878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.856188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.856439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] PATH_INFO: `/v3/users/31726794c42d4eb19f9ebedba8f8f35b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.857671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] RBAC: Authorizing `identity:get_user(user_id=31726794c42d4eb19f9ebedba8f8f35b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:41.859397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:41.860980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 154/307] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:41 2022] GET /identity/v3/users/31726794c42d4eb19f9ebedba8f8f35b => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:41.861739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:41.880145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:41.945227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c04ca6-829a-4e25-9aaf-b97c4df020a6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:41.954545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:41.967304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ba11f-c33f-4a93-9603-bae6de904484 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:41.979141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:41.979927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:41.980481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:41.980753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:41.982363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:41.984096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:41.985644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 154/308] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:41.986371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:42.001930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.002560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.002871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.003143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] PATH_INFO: `/v3/users/0f7c5ffbb97b43e7afeb0dd7c889a1da` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.016038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] RBAC: Authorizing `identity:get_user(user_id=0f7c5ffbb97b43e7afeb0dd7c889a1da)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:42.017472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:42.019109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 155/309] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:41 2022] GET /identity/v3/users/0f7c5ffbb97b43e7afeb0dd7c889a1da => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:42.019872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:42.088226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d5b81-d0ef-4068-9660-557b2f0e4260 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:42.092052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:42.116109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f1558-b4e0-4729-a4d3-6ba8930239c3 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:42.122133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.122857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.123140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.123415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] PATH_INFO: `/v3/system/users/31726794c42d4eb19f9ebedba8f8f35b/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.124811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=31726794c42d4eb19f9ebedba8f8f35b, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:42.126171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:42.148846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.149461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.149740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.150013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.160298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:42.165010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:42.166026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 156/310] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:42.167038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:42.191855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 155/311] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Aug 24 12:11:41 2022] PUT /identity/v3/system/users/31726794c42d4eb19f9ebedba8f8f35b/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:42.192872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:42.245024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:42.277878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6c92-fcc2-430f-bc7c-a624b68619d4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:42.313801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787b36e-78a2-414f-bf06-9c7cffcaa61e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:42.315693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.316315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.316589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.316854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] PATH_INFO: `/v3/groups` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.317499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:42.319129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:42.323622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:42.351213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.351818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.352094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.352367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.353094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] RBAC: Adding query filter params (role.id=2981ea0fce1c45c8b15e4bac0a72dd2f, user.id=0f7c5ffbb97b43e7afeb0dd7c889a1da, scope.system=all) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:42.353399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:42.354861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:42.360786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 156/312] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 12:11:42 2022] GET /identity/v3/role_assignments?user.id=0f7c5ffbb97b43e7afeb0dd7c889a1da&scope.system=all&role.id=2981ea0fce1c45c8b15e4bac0a72dd2f => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:42.364065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:42.407360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 157/313] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Aug 24 12:11:42 2022] POST /identity/v3/groups => generated 229 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:42.481082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfa662-e620-404a-9eaf-3fe4e663aea4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:42.484219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:42.521290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.521912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.522192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.522467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.523242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:42.523546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:42.525078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:42.537673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 157/314] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:42.543265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:42.656326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880504d3-4706-41c4-be16-182bf78b7d4c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:42.691305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.691915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.692241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.692522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] PATH_INFO: `/v3/users/4be25d53479248e4948b98b9a2fa0245` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.693667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] RBAC: Authorizing `identity:get_user(user_id=4be25d53479248e4948b98b9a2fa0245)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:42.695149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:42.696744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 158/315] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:42 2022] GET /identity/v3/users/4be25d53479248e4948b98b9a2fa0245 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:42.703210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:42.811386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b6c45-0a8c-46bd-8e7f-9d12bd9b446a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:42.846370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:42.847366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:42.847838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:42.848299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:42.884801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:42.922881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:42.924669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:43.024191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 158/316] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:43.030850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:43.095306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:43.139140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf1089-c38c-4499-9716-9879c926eca3 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:43.173882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8548b-7b01-4a76-b830-a57f845b34b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:43.174852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8548b-7b01-4a76-b830-a57f845b34b5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:43.175345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8548b-7b01-4a76-b830-a57f845b34b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:43.175824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8548b-7b01-4a76-b830-a57f845b34b5 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:43.177116 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8548b-7b01-4a76-b830-a57f845b34b5 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:43.178815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8548b-7b01-4a76-b830-a57f845b34b5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:43.180924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 159/317] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:43 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:43.186456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:43.259330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:43.290410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68c040-0583-4d16-b015-9cfafd0043e8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:43.330164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3578-5654-4232-b640-e91b8f31d4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:43.335515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3578-5654-4232-b640-e91b8f31d4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:43.336101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3578-5654-4232-b640-e91b8f31d4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:43.336580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3578-5654-4232-b640-e91b8f31d4c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:43.337619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a3578-5654-4232-b640-e91b8f31d4c1 None admin] RBAC: Adding query filter params (role.id=2981ea0fce1c45c8b15e4bac0a72dd2f, scope.project.id=a0c53322174e446daa46641a7229671f, user.id=4be25d53479248e4948b98b9a2fa0245) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:43.338157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a3578-5654-4232-b640-e91b8f31d4c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:43.340091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a3578-5654-4232-b640-e91b8f31d4c1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:43.347559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 159/318] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:11:43 2022] GET /identity/v3/role_assignments?user.id=4be25d53479248e4948b98b9a2fa0245&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=2981ea0fce1c45c8b15e4bac0a72dd2f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:44.190962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d629d7c-3272-46a7-95c4-41f64ff26772 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.191474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d629d7c-3272-46a7-95c4-41f64ff26772 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.191548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d629d7c-3272-46a7-95c4-41f64ff26772 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.192876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 160/319] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:44.198046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed40cb-92bc-4e06-ae24-c30a3a589555 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.202557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed40cb-92bc-4e06-ae24-c30a3a589555 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.202557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed40cb-92bc-4e06-ae24-c30a3a589555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.216921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0bed40cb-92bc-4e06-ae24-c30a3a589555 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:44.218108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0bed40cb-92bc-4e06-ae24-c30a3a589555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:44.286375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 160/320] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:44.318422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66318df-34e9-4966-a919-8ccacc00984d None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.318735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66318df-34e9-4966-a919-8ccacc00984d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.319011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66318df-34e9-4966-a919-8ccacc00984d None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.320467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 161/321] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:44.325322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befb05c-a48c-469e-b249-2f93e6615c87 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.325641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befb05c-a48c-469e-b249-2f93e6615c87 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.325909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befb05c-a48c-469e-b249-2f93e6615c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.344023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1befb05c-a48c-469e-b249-2f93e6615c87 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:44.345182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1befb05c-a48c-469e-b249-2f93e6615c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:44.400282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:44.411692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 161/322] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:44.421191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbecd6a-a8e2-436a-8490-abb81cc85b30 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.421481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbecd6a-a8e2-436a-8490-abb81cc85b30 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.424314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbecd6a-a8e2-436a-8490-abb81cc85b30 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.424314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 162/323] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:44.428541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd44d0-1972-4273-851e-3ea5f1644dda None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.428832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd44d0-1972-4273-851e-3ea5f1644dda None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.429108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd44d0-1972-4273-851e-3ea5f1644dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.447139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-67bd44d0-1972-4273-851e-3ea5f1644dda None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:44.448258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-67bd44d0-1972-4273-851e-3ea5f1644dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:44.514426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 162/324] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:44.556827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbecd6a-a8e2-436a-8490-abb81cc85b30 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:44.656584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbecd6a-a8e2-436a-8490-abb81cc85b30 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:44.680892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd44d0-1972-4273-851e-3ea5f1644dda None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:44.684302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:44.690741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:44.691397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.691671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.691937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.702431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:44.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:44.708249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:44.709038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 163/325] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:44.714165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:44.783647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd44d0-1972-4273-851e-3ea5f1644dda None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:44.803422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:44.817303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e25dc1-fea9-4ed2-a603-913681e242f0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:44.818523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:44.818523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.818523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.818810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.828415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:44.833216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:44.834079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:44.834936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 163/326] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:11:44.836556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:44.851527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:44.852124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.852398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.852661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.853319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:44.853612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:44.855008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:44.863847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 164/327] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:44.864622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:44.915218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:44.942423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3061a75-f450-41b7-8609-5fcd91eb82d5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:44.962695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c717258-44d1-4d5f-91bf-f0a4543b1b7c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:44.976201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:44.976822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.977097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.977369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.978485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:44.979964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:44.981527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 164/328] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:11:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:44.983120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:44.995213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:44.995822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:44.996098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:44.996366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:44.997028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:44.997404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:44.998804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:45.006618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 165/329] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:45.007392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:45.034869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:45.084630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece7c47-5280-4b1a-b7d4-1206a8357a08 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:45.107468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271e2af-574a-44b3-bf72-3bfd7e4b70ad None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:45.119491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:45.120223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:45.120507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:45.120781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] PATH_INFO: `/v3/users/31726794c42d4eb19f9ebedba8f8f35b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:45.121918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] RBAC: Authorizing `identity:get_user(user_id=31726794c42d4eb19f9ebedba8f8f35b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:45.122128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:45.123934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:45.125525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 165/330] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:44 2022] GET /identity/v3/users/31726794c42d4eb19f9ebedba8f8f35b => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:45.126287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:45.145849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:45.146725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:45.146998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:45.147263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] PATH_INFO: `/v3/groups` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:45.147874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:45.149310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:45.225116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795aa3e-6943-48e8-9507-190954946da3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:45.240337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 166/331] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Aug 24 12:11:45 2022] POST /identity/v3/groups => generated 226 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:11:45.241111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:45.241215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:45.268505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:45.269456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:45.269740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:45.270010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] PATH_INFO: `/v3/users/0f7c5ffbb97b43e7afeb0dd7c889a1da` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:45.271894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] RBAC: Authorizing `identity:get_user(user_id=0f7c5ffbb97b43e7afeb0dd7c889a1da)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:45.273365 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:45.279770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 166/332] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:45 2022] GET /identity/v3/users/0f7c5ffbb97b43e7afeb0dd7c889a1da => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:45.281197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:45.362061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e6b52-9e9e-4b31-9136-6956d5e4ca92 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:45.397624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d93fd-06a9-49ba-a0b4-7ab729a7a01e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:45.405593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6c52b-7456-4906-b6b2-5a41e9ecabe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:45.406208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6c52b-7456-4906-b6b2-5a41e9ecabe9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:45.406487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6c52b-7456-4906-b6b2-5a41e9ecabe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:45.406794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6c52b-7456-4906-b6b2-5a41e9ecabe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:45.407534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6c52b-7456-4906-b6b2-5a41e9ecabe9 None admin] RBAC: Adding query filter params (role.id=dbcd6da9a237441cb483965b19c21cff, user.id=31726794c42d4eb19f9ebedba8f8f35b, scope.system=all) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:45.407836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6c52b-7456-4906-b6b2-5a41e9ecabe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:45.409327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6c52b-7456-4906-b6b2-5a41e9ecabe9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:45.425180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 167/333] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 12:11:45 2022] GET /identity/v3/role_assignments?user.id=31726794c42d4eb19f9ebedba8f8f35b&scope.system=all&role.id=dbcd6da9a237441cb483965b19c21cff => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:45.456247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af95c95-f29b-45fa-9875-b5d9957b2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:45.457025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95c95-f29b-45fa-9875-b5d9957b2eb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:45.457378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95c95-f29b-45fa-9875-b5d9957b2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:45.457744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af95c95-f29b-45fa-9875-b5d9957b2eb8 None admin] PATH_INFO: `/v3/system/users/0f7c5ffbb97b43e7afeb0dd7c889a1da/roles/2981ea0fce1c45c8b15e4bac0a72dd2f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:45.459533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af95c95-f29b-45fa-9875-b5d9957b2eb8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f7c5ffbb97b43e7afeb0dd7c889a1da, role_id=2981ea0fce1c45c8b15e4bac0a72dd2f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:45.461084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af95c95-f29b-45fa-9875-b5d9957b2eb8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:45.521242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 167/334] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Aug 24 12:11:45 2022] PUT /identity/v3/system/users/0f7c5ffbb97b43e7afeb0dd7c889a1da/roles/2981ea0fce1c45c8b15e4bac0a72dd2f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:47.385424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f93920-1a4a-4166-b839-81a2fa8356b1 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:47.386496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f93920-1a4a-4166-b839-81a2fa8356b1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:47.387016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f93920-1a4a-4166-b839-81a2fa8356b1 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:47.388999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 168/335] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:47.394761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9782f-4f73-4755-bb94-4591345172a5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:47.395600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9782f-4f73-4755-bb94-4591345172a5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:47.396122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9782f-4f73-4755-bb94-4591345172a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:47.419062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-03e9782f-4f73-4755-bb94-4591345172a5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:47.421099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-03e9782f-4f73-4755-bb94-4591345172a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:47.483721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:47.511207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 168/336] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:47.757082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a648dce-21b8-49e8-91b8-d6b3f5b23eb1 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:47.757722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a648dce-21b8-49e8-91b8-d6b3f5b23eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:47.758221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a648dce-21b8-49e8-91b8-d6b3f5b23eb1 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:47.760026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 169/337] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:47.776408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada064-20b6-454a-a1b4-35177d204fc1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:47.776706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada064-20b6-454a-a1b4-35177d204fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:47.776969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada064-20b6-454a-a1b4-35177d204fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:47.801133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c5ada064-20b6-454a-a1b4-35177d204fc1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:47.802921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c5ada064-20b6-454a-a1b4-35177d204fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:47.909378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 169/338] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:47.923705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a648dce-21b8-49e8-91b8-d6b3f5b23eb1 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:47.983267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:48.033711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f7d3-43ef-454e-bb5c-6a0b34e137ed None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.034390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f7d3-43ef-454e-bb5c-6a0b34e137ed None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.034390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f7d3-43ef-454e-bb5c-6a0b34e137ed None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.037190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 170/339] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:48.042893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21438c0-67ad-4545-9f62-9521a684dd37 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.043273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21438c0-67ad-4545-9f62-9521a684dd37 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.049269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21438c0-67ad-4545-9f62-9521a684dd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.076892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f21438c0-67ad-4545-9f62-9521a684dd37 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:48.080969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f21438c0-67ad-4545-9f62-9521a684dd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:48.082309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a648dce-21b8-49e8-91b8-d6b3f5b23eb1 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:48.164253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c9638-4c1e-4685-b012-fad7afdd6b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:48.165296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c9638-4c1e-4685-b012-fad7afdd6b64 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.165758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c9638-4c1e-4685-b012-fad7afdd6b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.166251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c9638-4c1e-4685-b012-fad7afdd6b64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.195387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c9638-4c1e-4685-b012-fad7afdd6b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-3f8c9638-4c1e-4685-b012-fad7afdd6b64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:48.205532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:48.208299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:48.209095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:48.209698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 170/340] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:48.210681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7eed7-fe23-41fb-a650-e4e25e2bc795 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.210968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7eed7-fe23-41fb-a650-e4e25e2bc795 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.211237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7eed7-fe23-41fb-a650-e4e25e2bc795 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.213357 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 171/341] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:48.214061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 171/342] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:48.221117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20910754-07b6-49ca-b9b8-935eee374936 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.221398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20910754-07b6-49ca-b9b8-935eee374936 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.221671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20910754-07b6-49ca-b9b8-935eee374936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.226634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7eed7-fe23-41fb-a650-e4e25e2bc795 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:48.258322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-20910754-07b6-49ca-b9b8-935eee374936 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:48.262083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-20910754-07b6-49ca-b9b8-935eee374936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:48.322504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:48.369693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 172/343] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:48.370605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20910754-07b6-49ca-b9b8-935eee374936 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:48.380806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:48.399867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7eed7-fe23-41fb-a650-e4e25e2bc795 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:48.464247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:48.465129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.465568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.466033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.466867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:48.467312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:48.468821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:48.482589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 172/344] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:48.507696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:48.510523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20910754-07b6-49ca-b9b8-935eee374936 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:48.567381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:48.568284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.568704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.569104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.588362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:48.598864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:48.599817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:48.600599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 173/345] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:48.607130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:48.641496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e4d6f-d752-45a4-8aba-12106a769633 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:48.678688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:48.694176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:48.695138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.695596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.696139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.738695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:48.766895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23cfd9-3051-4039-8302-4649bf557027 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:48.786337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:48.788154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:48.840260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:48.841719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:48.842177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:48.842621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:48.843485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:48.843976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:48.846990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:48.872929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 174/346] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:48.899934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:48.933588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 173/347] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:48.934734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:48.997035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.030712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.040092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24621963-d14d-43ab-8aa3-535ebb7b8b29 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.048582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e11a90-6290-4865-a94d-b6ebf371a272 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.078996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.079623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.079903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.080175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] PATH_INFO: `/v3/users/0f7c5ffbb97b43e7afeb0dd7c889a1da` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.081589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] RBAC: Authorizing `identity:get_user(user_id=0f7c5ffbb97b43e7afeb0dd7c889a1da)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.083006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:49.085036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 175/348] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:11:48 2022] GET /identity/v3/users/0f7c5ffbb97b43e7afeb0dd7c889a1da => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:49.089743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.093226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.093822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.094102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.094371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.110714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:49.116256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:49.117150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:49.117964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:49.120611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 174/349] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:49.121742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.210267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811653c-2d52-40f2-98fc-a7590754df54 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.248942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a87de-5432-4442-bdf5-05d4effd1afe None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.250996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.251841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.252184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.252526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.255509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.256078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:49.263707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 176/350] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:49.264205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.269550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.293675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.294326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.294603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.294904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.296837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.306681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:49.312498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:49.313388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:49.314181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:49.318849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 175/351] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:49.320684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.398858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04164ec-4ad9-4ddc-ad19-b20c18a70497 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.449433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcb9f2-bb0b-40ed-9115-3d30fc98112f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.452921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92340c47-a735-4b40-b946-7163c9707921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.453525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92340c47-a735-4b40-b946-7163c9707921 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.453801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92340c47-a735-4b40-b946-7163c9707921 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.454068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92340c47-a735-4b40-b946-7163c9707921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.454825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92340c47-a735-4b40-b946-7163c9707921 None admin] RBAC: Adding query filter params (role.id=2981ea0fce1c45c8b15e4bac0a72dd2f, user.id=0f7c5ffbb97b43e7afeb0dd7c889a1da, scope.system=all) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:49.455133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92340c47-a735-4b40-b946-7163c9707921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.456549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92340c47-a735-4b40-b946-7163c9707921 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:49.470490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 177/352] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/role_assignments?user.id=0f7c5ffbb97b43e7afeb0dd7c889a1da&scope.system=all&role.id=2981ea0fce1c45c8b15e4bac0a72dd2f => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:49.472093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ba69b-8488-408d-be7b-86be6b9e1b87 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.472613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ba69b-8488-408d-be7b-86be6b9e1b87 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.473077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ba69b-8488-408d-be7b-86be6b9e1b87 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.475058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 178/353] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:49.476096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ba69b-8488-408d-be7b-86be6b9e1b87 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.495228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.496065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.496346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.496622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.498581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:49.498842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.500600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:49.513083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 176/354] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:49.513810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.571054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.600015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.623922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ba69b-8488-408d-be7b-86be6b9e1b87 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.657979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a98d8-4e3e-4fa3-a0ba-0669d0595ad4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.665274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb26d54-d91d-4029-afb7-b72a8fa0236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.666291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb26d54-d91d-4029-afb7-b72a8fa0236b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.666566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb26d54-d91d-4029-afb7-b72a8fa0236b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.666865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb26d54-d91d-4029-afb7-b72a8fa0236b None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.674989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb26d54-d91d-4029-afb7-b72a8fa0236b None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.676421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb26d54-d91d-4029-afb7-b72a8fa0236b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:49.678079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 179/355] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:49.680356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433be42-d26b-4f7e-8691-cd020203af0d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.680642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433be42-d26b-4f7e-8691-cd020203af0d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.680917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433be42-d26b-4f7e-8691-cd020203af0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.701542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f433be42-d26b-4f7e-8691-cd020203af0d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:49.702773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f433be42-d26b-4f7e-8691-cd020203af0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:49.707607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.708210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.708500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.708747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.709435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:49.709731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.712200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:49.724112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 177/356] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:49.726726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.792458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 180/357] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:49.793582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433be42-d26b-4f7e-8691-cd020203af0d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.866727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.870495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac68df6-9978-4547-8b52-1ff506a86be3 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.912794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:49.916424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ab575-1fe6-4fb5-8e04-e4e95ba318e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.917034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ab575-1fe6-4fb5-8e04-e4e95ba318e8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.917312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ab575-1fe6-4fb5-8e04-e4e95ba318e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.917582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ab575-1fe6-4fb5-8e04-e4e95ba318e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.936287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433be42-d26b-4f7e-8691-cd020203af0d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:49.973396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:49.974093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:49.974374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:49.974640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:49.975445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=c53f8195f07e4713b63a46a6129a59d1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:49.975752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.977444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:49.983509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 181/358] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=dbcd6da9a237441cb483965b19c21cff => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:49.984296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:49.995058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ab575-1fe6-4fb5-8e04-e4e95ba318e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:49.996899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ab575-1fe6-4fb5-8e04-e4e95ba318e8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:50.096349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e54c1-8c3f-49a1-8d9e-4f197169d5e7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:50.108661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:50.114705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 178/359] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:50.116330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485f80-58a0-41be-aab6-c3d64613302c None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.116858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485f80-58a0-41be-aab6-c3d64613302c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.117351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485f80-58a0-41be-aab6-c3d64613302c None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.122780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 179/360] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:50.122780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df485f80-58a0-41be-aab6-c3d64613302c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:50.141835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33df3d0-fb38-4a6b-a8f3-8e4b5d59e8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:50.142417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33df3d0-fb38-4a6b-a8f3-8e4b5d59e8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.142719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33df3d0-fb38-4a6b-a8f3-8e4b5d59e8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.143823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33df3d0-fb38-4a6b-a8f3-8e4b5d59e8d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.162390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:50.236044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33df3d0-fb38-4a6b-a8f3-8e4b5d59e8d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:50.238368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33df3d0-fb38-4a6b-a8f3-8e4b5d59e8d1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:50.288430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df485f80-58a0-41be-aab6-c3d64613302c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:50.332371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:50.333776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.334056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.334326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.349577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:50.354529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:50.355491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:50.356358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 180/361] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:50.357562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:50.383991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 182/362] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:50.386437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46ddb-3622-440f-acd4-cf095745bf7e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.386761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46ddb-3622-440f-acd4-cf095745bf7e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.387038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46ddb-3622-440f-acd4-cf095745bf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.413022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-90f46ddb-3622-440f-acd4-cf095745bf7e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:50.414563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-90f46ddb-3622-440f-acd4-cf095745bf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:50.435996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:50.459650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:50.493124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a5551-c496-452d-84e0-d5f461745876 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:50.503415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 183/363] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:50.503834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f46ddb-3622-440f-acd4-cf095745bf7e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:50.541217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:50.542138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.542419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.543111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.545117 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:50.547799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:50.549582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 181/364] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:50 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:50.550440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:50.644676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f46ddb-3622-440f-acd4-cf095745bf7e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:50.687187 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:50.687903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.688185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.688561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.689233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:50.689532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:50.692582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:50.696989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c95d6-f42b-4945-8211-2b4569a35d1e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:50.702317 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 184/365] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:50.703192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:50.737555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:50.756437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:50.756437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.756437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.756437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.756799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:50.756799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:50.756799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 182/366] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:50 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:50.756799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:50.758031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:50.836700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc486c-b85a-4351-a2ec-4930b7bcab9a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:50.888284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:50.888924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.889203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.889300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b337a5-9905-4ab7-9ff2-79c03e3dfce5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:50.889472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:50.891360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:50.892919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:50.898675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 185/367] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:50 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:50.898675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:50.932657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:50.933273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:50.933551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:50.933824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.015962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:51.027390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4b67-adb2-4986-b390-18fa42a65b53 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.032824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:51.032960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.036297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.077162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.078145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.078636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.079129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.080452 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.081886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.084791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 186/368] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:50 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:51.084791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.175221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 183/369] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:51.180840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.206456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70df97a-b69e-4ff1-8b4d-21b9beb920df None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.254720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.255513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.255881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.256256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.268076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:51.274128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:51.275181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:51.275974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 187/370] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:51.276626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.294517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:51.301429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64524a4-783f-4544-bfb3-aac4750e033f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.316373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:51.344294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.345311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.345655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.345937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.346744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=a0c53322174e446daa46641a7229671f) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:51.347058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.348699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.358748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 184/371] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=dbcd6da9a237441cb483965b19c21cff => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:51.358748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.409608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58aa221-352b-4a23-9c53-40dbddb357b2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.456810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.457426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.457701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.457970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.458798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=c53f8195f07e4713b63a46a6129a59d1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:51.459108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.460574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.468610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 188/372] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:51.469903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.497944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ae196-ad92-4eed-a418-5b4dd8b620bf None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.541415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.542056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.542333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.542711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.544459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.545923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.548253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 185/373] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:51.553878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.598297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:51.605882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:51.615266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee0603-29d9-4163-ac84-2a82df0f5c28 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.658795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.659572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.659981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.660358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.661252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:51.661660 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.663215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.677882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 189/374] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:51.678822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebef23f-a00d-4948-bc27-fbba77c4c0f4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.683660 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.717844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.718909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.719427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.719892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.721259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.722972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.725079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 186/375] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:51.737217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:51.841520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d6213-924f-4ee8-9b5c-ae2344f5d42a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.849788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:51.861355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea5219-236b-48b4-82e4-ef9bd987707a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:51.902430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b976d-76d5-4f2b-bebd-1fe868706909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.903065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b976d-76d5-4f2b-bebd-1fe868706909 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.903985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b976d-76d5-4f2b-bebd-1fe868706909 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.904259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b976d-76d5-4f2b-bebd-1fe868706909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.905066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b976d-76d5-4f2b-bebd-1fe868706909 None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=a0c53322174e446daa46641a7229671f) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:51.905379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b976d-76d5-4f2b-bebd-1fe868706909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:51.906944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b976d-76d5-4f2b-bebd-1fe868706909 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:51.918345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 187/376] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:51.927080 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:51.927716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:51.927999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:51.928277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:51.975370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:52.024350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:52.025830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:52.097927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ec406-4674-41f5-b725-4214f9684227 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:52.098255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ec406-4674-41f5-b725-4214f9684227 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:52.098409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ec406-4674-41f5-b725-4214f9684227 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:52.100394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 188/377] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:52.112813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be92d3-c8b0-4dfc-bf4c-3d81265cf4a0 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:52.113097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be92d3-c8b0-4dfc-bf4c-3d81265cf4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:52.113368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be92d3-c8b0-4dfc-bf4c-3d81265cf4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:52.147618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 190/378] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:11:52.147814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-41be92d3-c8b0-4dfc-bf4c-3d81265cf4a0 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:52.149084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-41be92d3-c8b0-4dfc-bf4c-3d81265cf4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:52.154123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:52.250952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:52.275146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 189/379] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:52.306877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a5655-5063-4f8b-9af6-45c46ccbc91f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:52.349528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:52.350072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:52.350351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:52.350621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] PATH_INFO: `/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:52.352412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] RBAC: Authorizing `identity:get_project(project_id=4cfd603abe4c44b2a5789dbc4d8a6f2b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:52.353927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:52.356020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 191/380] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:52 2022] GET /identity/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:52.361654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be92d3-c8b0-4dfc-bf4c-3d81265cf4a0 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:52.493705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:52.509510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be92d3-c8b0-4dfc-bf4c-3d81265cf4a0 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:52.546030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:52.546674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:52.547032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:52.547302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] PATH_INFO: `/v3/groups/f545c5fb848f4786aead9ee87b091c6a` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:52.555957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] RBAC: Authorizing `identity:get_group(group_id=f545c5fb848f4786aead9ee87b091c6a)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:52.557401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:52.559983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 190/381] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:52 2022] GET /identity/v3/groups/f545c5fb848f4786aead9ee87b091c6a => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:52.565457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:52.704239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:52.707014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54b181-6363-477d-8f3e-2b0670dad48b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:52.738586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:52.751756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:52.763832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:52.763832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:52.763832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:52.763832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] RBAC: Adding query filter params (group.id=f545c5fb848f4786aead9ee87b091c6a, role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=4cfd603abe4c44b2a5789dbc4d8a6f2b) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:52.763832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:52.763832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:52.764269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 192/382] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:52 2022] GET /identity/v3/role_assignments?group.id=f545c5fb848f4786aead9ee87b091c6a&scope.project.id=4cfd603abe4c44b2a5789dbc4d8a6f2b&role.id=753fd078fdac40abb65aa43057d79c62 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:52.890617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839bec5-2b8b-40a3-886e-e79e9253ddd1 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:52.935485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:52.936112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:52.936390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:52.936660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:52.949291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:52.955697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:52.957156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:52.958288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:52.960350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 191/383] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:11:52.967768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:53.094866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb1680-ab67-486b-8c3b-d45b3c01df77 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:53.135396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:53.136001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:53.136277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:53.136546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:53.137203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:53.138039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:53.139441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:53.154099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 193/384] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:53.159822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:53.185755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:53.309914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaf4cd-8058-429e-a479-ceffb667787f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:53.357326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:53.358565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:53.359049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:53.359500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:53.360799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:53.362430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:53.364222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 192/385] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:53 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:53.369958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:53.456738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:53.517195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dd7e1-ea1d-4e9c-82b5-8118eab9a101 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:53.559101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f66c2c-f007-40bf-8351-2954d2c3d9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:53.560345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66c2c-f007-40bf-8351-2954d2c3d9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:53.560560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66c2c-f007-40bf-8351-2954d2c3d9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:53.560828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66c2c-f007-40bf-8351-2954d2c3d9f6 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:53.565108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f66c2c-f007-40bf-8351-2954d2c3d9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:53.566607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f66c2c-f007-40bf-8351-2954d2c3d9f6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:53.568287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 194/386] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:53 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:53.573979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:53.696656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:53.720852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc832971-f943-4b9d-b3a1-2fdd512a4129 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:53.766327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d6c8f-d326-4635-af47-48b5c30d1a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:53.767010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d6c8f-d326-4635-af47-48b5c30d1a41 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:53.767289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d6c8f-d326-4635-af47-48b5c30d1a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:53.767563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d6c8f-d326-4635-af47-48b5c30d1a41 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1/groups/150613bd47594e6e8fec972e369754b1/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:53.769493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d6c8f-d326-4635-af47-48b5c30d1a41 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53f8195f07e4713b63a46a6129a59d1, group_id=150613bd47594e6e8fec972e369754b1, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:53.770960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d6c8f-d326-4635-af47-48b5c30d1a41 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:53.848557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 193/387] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 12:11:53 2022] PUT /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1/groups/150613bd47594e6e8fec972e369754b1/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:54.016055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aacdfa-57ed-42f5-8ac3-5ff5ca599ef5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.016569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aacdfa-57ed-42f5-8ac3-5ff5ca599ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.016569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aacdfa-57ed-42f5-8ac3-5ff5ca599ef5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.018216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 195/388] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:54.024264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a0506-8a32-41d2-8a32-a56915f841fc None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.024407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076315f9-9384-4340-8007-963fea148619 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.024589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a0506-8a32-41d2-8a32-a56915f841fc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.024683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076315f9-9384-4340-8007-963fea148619 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.024849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a0506-8a32-41d2-8a32-a56915f841fc None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.024938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076315f9-9384-4340-8007-963fea148619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.026082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 196/389] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:11:54.039039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1188416-46e1-410b-8987-8a3a9cff12dd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.039342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1188416-46e1-410b-8987-8a3a9cff12dd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.040177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1188416-46e1-410b-8987-8a3a9cff12dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.045267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-076315f9-9384-4340-8007-963fea148619 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:54.046470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-076315f9-9384-4340-8007-963fea148619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:54.066977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d1188416-46e1-410b-8987-8a3a9cff12dd None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:54.068201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d1188416-46e1-410b-8987-8a3a9cff12dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:54.146584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 194/390] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:54.170992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 197/391] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:54.420623 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076315f9-9384-4340-8007-963fea148619 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:54.511159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:54.524977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1188416-46e1-410b-8987-8a3a9cff12dd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:54.557745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076315f9-9384-4340-8007-963fea148619 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:54.588256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:54.593775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ffcaa-0927-4b95-a7d4-eaf160e03648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:54.594405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ffcaa-0927-4b95-a7d4-eaf160e03648 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.594704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ffcaa-0927-4b95-a7d4-eaf160e03648 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.594988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ffcaa-0927-4b95-a7d4-eaf160e03648 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.605015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ffcaa-0927-4b95-a7d4-eaf160e03648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-b01ffcaa-0927-4b95-a7d4-eaf160e03648 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:54.612463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:54.613591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:54.614613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:54.616960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 195/392] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:54.618318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97673-1293-49ca-ab96-452f3ece602b None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.618781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97673-1293-49ca-ab96-452f3ece602b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.619200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97673-1293-49ca-ab96-452f3ece602b None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.620866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 196/393] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:54.626541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97673-1293-49ca-ab96-452f3ece602b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:54.658607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1188416-46e1-410b-8987-8a3a9cff12dd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:54.716272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87190ff-0358-4c6b-a91d-87426454272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:54.717703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87190ff-0358-4c6b-a91d-87426454272f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.718484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87190ff-0358-4c6b-a91d-87426454272f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.719002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87190ff-0358-4c6b-a91d-87426454272f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.735111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87190ff-0358-4c6b-a91d-87426454272f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:54.740584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97673-1293-49ca-ab96-452f3ece602b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e87190ff-0358-4c6b-a91d-87426454272f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:54.742141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:54.743051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:54.743838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:54.747506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 198/394] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:54.748968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a321ed-485c-436f-8382-7b51bd710262 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.749413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a321ed-485c-436f-8382-7b51bd710262 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.749829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a321ed-485c-436f-8382-7b51bd710262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.773069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:54.776563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-42a321ed-485c-436f-8382-7b51bd710262 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:54.778011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-42a321ed-485c-436f-8382-7b51bd710262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:54.788754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:54.789682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.790710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.791206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.792796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:54.793346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:54.796209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:54.810051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 197/395] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:54.811456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:54.861474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 199/396] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:54.868208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a321ed-485c-436f-8382-7b51bd710262 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:54.915028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:54.947564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd867d-365e-4f74-8272-6b8592306833 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:54.986769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a321ed-485c-436f-8382-7b51bd710262 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:54.996075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:54.996705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:54.996988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:54.997257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:54.998278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:54.998395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:54.999712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.020672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 198/397] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:55.024544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.025843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.026183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.026644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.027335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.029270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.031497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.033365 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 200/398] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:54 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:55.044493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.145744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:55.154749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4ca84-73c3-426b-9d9a-4386431985c2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.167033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:55.194527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.195412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.203370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.203370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.203370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.203370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.203370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 201/399] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:55 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:11:55.206826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.253389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72860e04-f360-4f1a-9894-77f757305389 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.290066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.291894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.292188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.292466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.294545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.295658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.297240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 199/400] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:55 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:55.298148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.331656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d46bf-0433-4ece-b6c5-31e3fec75bdb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.368454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4681e-fdcb-4d03-a132-66b3bb34f372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.369114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4681e-fdcb-4d03-a132-66b3bb34f372 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.369393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4681e-fdcb-4d03-a132-66b3bb34f372 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.369667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4681e-fdcb-4d03-a132-66b3bb34f372 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1/groups/150613bd47594e6e8fec972e369754b1/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.371887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4681e-fdcb-4d03-a132-66b3bb34f372 None admin] RBAC: Authorizing `identity:create_grant(project_id=c53f8195f07e4713b63a46a6129a59d1, group_id=150613bd47594e6e8fec972e369754b1, role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.373315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4681e-fdcb-4d03-a132-66b3bb34f372 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.405915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c2635-f7af-4442-912b-941ad9fb4e11 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.430495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:55.447550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 202/401] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 12:11:55 2022] PUT /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1/groups/150613bd47594e6e8fec972e369754b1/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:55.447550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059717b-95b3-4cf2-b90c-f915f1869da7 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.447550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059717b-95b3-4cf2-b90c-f915f1869da7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.447550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059717b-95b3-4cf2-b90c-f915f1869da7 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.447550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 203/402] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:55.447550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059717b-95b3-4cf2-b90c-f915f1869da7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.453467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d5c3-7512-440f-a0dc-d8018b169594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.453682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d5c3-7512-440f-a0dc-d8018b169594 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.453855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d5c3-7512-440f-a0dc-d8018b169594 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.454121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d5c3-7512-440f-a0dc-d8018b169594 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.468823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9d5c3-7512-440f-a0dc-d8018b169594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-fda9d5c3-7512-440f-a0dc-d8018b169594 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:55.474502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:55.475492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:55.476288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 200/403] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:55.477455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924ee20-f68d-4a03-b018-4897ca8d6cd6 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.477455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924ee20-f68d-4a03-b018-4897ca8d6cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.477775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924ee20-f68d-4a03-b018-4897ca8d6cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.500798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a924ee20-f68d-4a03-b018-4897ca8d6cd6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:55.501016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:55.502783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a924ee20-f68d-4a03-b018-4897ca8d6cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:55.593982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 201/404] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:55.600332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924ee20-f68d-4a03-b018-4897ca8d6cd6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.603506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059717b-95b3-4cf2-b90c-f915f1869da7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.646876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.647881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.648361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.649161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.650912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.652983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.655839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 204/405] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:55 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:55.661969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.717414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924ee20-f68d-4a03-b018-4897ca8d6cd6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.732216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:55.756486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.757238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.757600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.757934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.758740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:55.759110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.760719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.771759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 202/406] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:55.783795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.788136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221b49-4105-4c29-be0e-2b9d9832cf7f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.796090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:55.835475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.836390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.837125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.837568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f/groups/150613bd47594e6e8fec972e369754b1/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.839927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c53322174e446daa46641a7229671f, group_id=150613bd47594e6e8fec972e369754b1, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.841830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.930753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736dd90-dc91-44a6-ac50-0064e1cfa03a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:55.956535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 205/407] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 12:11:55 2022] PUT /identity/v3/projects/a0c53322174e446daa46641a7229671f/groups/150613bd47594e6e8fec972e369754b1/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:11:55.956535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:55.988715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:55.989355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:55.989632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:55.990114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:55.992681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:55.995115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:55.999930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 203/408] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:55 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:56.005758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.086615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:56.162321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9521a-b843-4e45-bccc-8711e2bffd0b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:56.180859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bedded-189f-46dd-b7da-c7588e3eada4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:56.215953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874167-2977-4025-a9ea-78bfb90d127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:56.216925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874167-2977-4025-a9ea-78bfb90d127d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.217360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874167-2977-4025-a9ea-78bfb90d127d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.217796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874167-2977-4025-a9ea-78bfb90d127d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.228892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874167-2977-4025-a9ea-78bfb90d127d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-7d874167-2977-4025-a9ea-78bfb90d127d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:56.234734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:56.235647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:56.236849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 206/409] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:11:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:56.237510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbfa16-1de4-4cea-bd99-27b18b7dd47b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.237583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbfa16-1de4-4cea-bd99-27b18b7dd47b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.237898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbfa16-1de4-4cea-bd99-27b18b7dd47b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.242701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 207/410] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:56.243520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c0c9c-a437-4652-bb2f-4a6440deb88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:56.244337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c0c9c-a437-4652-bb2f-4a6440deb88d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.244759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c0c9c-a437-4652-bb2f-4a6440deb88d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.245189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c0c9c-a437-4652-bb2f-4a6440deb88d None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.247442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c0c9c-a437-4652-bb2f-4a6440deb88d None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:56.248386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbfa16-1de4-4cea-bd99-27b18b7dd47b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.250478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c0c9c-a437-4652-bb2f-4a6440deb88d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:56.253850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 204/411] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:56 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:56.255246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013468b3-11a0-4add-bf5b-11a1768f2c6f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.255634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013468b3-11a0-4add-bf5b-11a1768f2c6f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.256013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013468b3-11a0-4add-bf5b-11a1768f2c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.277990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-013468b3-11a0-4add-bf5b-11a1768f2c6f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:56.279461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-013468b3-11a0-4add-bf5b-11a1768f2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:56.284886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:56.369612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbfa16-1de4-4cea-bd99-27b18b7dd47b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:56.378726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 205/412] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:56.379805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013468b3-11a0-4add-bf5b-11a1768f2c6f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.437481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:56.438165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.438450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.438740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.439410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:56.439766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:56.441140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:56.461166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 208/413] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:56.467354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:56.470482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.549563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013468b3-11a0-4add-bf5b-11a1768f2c6f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:56.616849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:56.618306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.618822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.619320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f/groups/150613bd47594e6e8fec972e369754b1/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.622272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c53322174e446daa46641a7229671f, group_id=150613bd47594e6e8fec972e369754b1, role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:56.623945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:56.638990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:56.648392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6a0c7-67a3-4c26-958e-4c3abf829a2c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:56.683344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:56.685703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.685703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.685703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] PATH_INFO: `/v3/groups/f545c5fb848f4786aead9ee87b091c6a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.686205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] RBAC: Authorizing `identity:get_group(group_id=f545c5fb848f4786aead9ee87b091c6a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:56.687753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:56.689409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 209/414] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:56 2022] GET /identity/v3/groups/f545c5fb848f4786aead9ee87b091c6a => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:56.690092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.710259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 206/415] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 12:11:56 2022] PUT /identity/v3/projects/a0c53322174e446daa46641a7229671f/groups/150613bd47594e6e8fec972e369754b1/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:56.714384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.890191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f47fa-3bab-4172-a9f5-cfafcee39832 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:56.893393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:56.906103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5935bc-a5ea-4f7b-ad34-d8efb4adec32 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:56.938487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:56.939416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.939892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.940337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.948553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:56.949335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:56.949945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:56.950290 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] PATH_INFO: `/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:56.952006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] RBAC: Authorizing `identity:get_project(project_id=4cfd603abe4c44b2a5789dbc4d8a6f2b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:56.953393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:56.955113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:56.955822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 207/416] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:56 2022] GET /identity/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:56.960645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:56.961515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:56.962481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:11:56.963165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.964824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 210/417] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:56.970876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:56.989970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:57.092774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b222e-be8f-4275-b0f5-d70d6b0ecfda None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:57.128612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ade60c-2acc-4887-818b-f555de6e8313 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:57.129738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:57.130745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:57.131032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:57.131306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] PATH_INFO: `/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b/groups/f545c5fb848f4786aead9ee87b091c6a/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:57.133184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=4cfd603abe4c44b2a5789dbc4d8a6f2b, group_id=f545c5fb848f4786aead9ee87b091c6a, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:57.134614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:57.176682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:57.177400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:57.177634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:57.177908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:57.178621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] RBAC: Adding query filter params (name=member) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:57.178940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:57.184263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:57.189682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 211/418] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:57.195775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:57.209151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 208/419] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 12:11:56 2022] PUT /identity/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b/groups/f545c5fb848f4786aead9ee87b091c6a/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:11:57.304541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:57.337059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e81ae-d55b-43f3-89ea-062a70dc405c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:57.379382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:57.380010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:57.380290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:57.380569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:57.460319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:57.461707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:57.544534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:57.571101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 212/420] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:11:57.577725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:57.617657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:57.696772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc163a-e3cb-4057-bb68-d74dcf527dcd None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:57.732285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6286ad-d3e7-4285-ae9b-6622bb578bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:57.733178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6286ad-d3e7-4285-ae9b-6622bb578bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:57.733450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6286ad-d3e7-4285-ae9b-6622bb578bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:57.733717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6286ad-d3e7-4285-ae9b-6622bb578bb0 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:57.734854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6286ad-d3e7-4285-ae9b-6622bb578bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:57.736280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6286ad-d3e7-4285-ae9b-6622bb578bb0 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:57.738213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 209/421] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:57 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:57.743757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:57.912616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aedb2c-2955-4858-ae14-d11111fcc51c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:57.915751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515536-3e72-449b-9805-7a318e3b9c19 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:57.916037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515536-3e72-449b-9805-7a318e3b9c19 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:57.916363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515536-3e72-449b-9805-7a318e3b9c19 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:57.917850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 210/422] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:57.923723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ceec-9e83-490b-9534-220a6aa9d0af None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:57.924384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ceec-9e83-490b-9534-220a6aa9d0af None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:57.924856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ceec-9e83-490b-9534-220a6aa9d0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:57.946407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1bc3ceec-9e83-490b-9534-220a6aa9d0af None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:57.948921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1bc3ceec-9e83-490b-9534-220a6aa9d0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:57.972839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:57.973933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:57.974418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:57.974920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:57.977634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:57.979327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:57.983656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 213/423] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:57 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:57.989186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:58.066989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:58.079135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 211/424] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:11:58.153838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da2f1a-e00a-47ea-b849-042f0e6f7dd7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:58.202049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e2651-0238-47c3-9c97-722fb1605c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:58.203472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2651-0238-47c3-9c97-722fb1605c3a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:58.203957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2651-0238-47c3-9c97-722fb1605c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:58.204445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2651-0238-47c3-9c97-722fb1605c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:58.205463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2651-0238-47c3-9c97-722fb1605c3a None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=c53f8195f07e4713b63a46a6129a59d1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:58.205987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2651-0238-47c3-9c97-722fb1605c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:58.207676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2651-0238-47c3-9c97-722fb1605c3a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:58.219375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 214/425] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:57 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=dbcd6da9a237441cb483965b19c21cff => generated 611 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:58.285860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a95cc-4d13-434c-b824-bd61fc2b170f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:58.286515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a95cc-4d13-434c-b824-bd61fc2b170f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:58.287034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a95cc-4d13-434c-b824-bd61fc2b170f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:58.288661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 212/426] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:58.301356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8503-8505-4800-8d71-b7f54ba3c491 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:58.302255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8503-8505-4800-8d71-b7f54ba3c491 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:58.302960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8503-8505-4800-8d71-b7f54ba3c491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:58.327587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e92f8503-8505-4800-8d71-b7f54ba3c491 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:58.329269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e92f8503-8505-4800-8d71-b7f54ba3c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:58.418339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 215/427] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:58.441202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a95cc-4d13-434c-b824-bd61fc2b170f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:58.470340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:58.556507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a95cc-4d13-434c-b824-bd61fc2b170f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:58.594356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:58.595098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:58.595374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:58.595645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:58.606162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:58.611328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:58.612202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:58.613028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 213/428] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:58.618583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8503-8505-4800-8d71-b7f54ba3c491 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:58.657367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:58.731816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8503-8505-4800-8d71-b7f54ba3c491 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:58.738250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:58.773396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:58.774236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:58.774831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:58.775112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:58.775777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:58.776127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:58.777536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:58.786496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 216/429] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:11:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:58.792166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:58.879939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:58.892426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed231cf-51d1-45a0-ae29-4d2d79eef3f7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:58.929785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:58.935615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f51f5-f20e-4897-9979-d3c09130841b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:58.937250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:58.937395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:58.938437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:58.938437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:58.950962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:11:58.955574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:11:58.956462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:11:58.957280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 214/430] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:11:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:11:58.964140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:58.978582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:58.979225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:58.979507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:58.979784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.058130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.059682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.084995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ae5e1-3b54-4e0b-9a0d-8419d7b66c97 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:59.128662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:59.129637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.130230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.130766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.131660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] RBAC: Adding query filter params (name=member) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:59.132166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.133752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.145068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 215/431] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:11:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:59.150840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.163052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:59.164602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:59.169606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 217/432] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:11:59.177330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.274513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4f418-2618-409c-84fb-04fff2ba755c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:59.297378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8520a-6472-43c8-a0ad-d831f9a46a07 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:59.308579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f9319-8351-4d30-b5cc-12fbc3427da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:59.309852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f9319-8351-4d30-b5cc-12fbc3427da6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.310135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f9319-8351-4d30-b5cc-12fbc3427da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.310403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f9319-8351-4d30-b5cc-12fbc3427da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.335652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a4087-3221-4afd-8352-03aa1c8044a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:59.336663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a4087-3221-4afd-8352-03aa1c8044a9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.337225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a4087-3221-4afd-8352-03aa1c8044a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.337661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a4087-3221-4afd-8352-03aa1c8044a9 None admin] PATH_INFO: `/v3/projects/c53f8195f07e4713b63a46a6129a59d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.338985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a4087-3221-4afd-8352-03aa1c8044a9 None admin] RBAC: Authorizing `identity:get_project(project_id=c53f8195f07e4713b63a46a6129a59d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.340849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a4087-3221-4afd-8352-03aa1c8044a9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.342871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 218/433] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/projects/c53f8195f07e4713b63a46a6129a59d1 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:59.344142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e5b4f-0860-4dce-8a76-ffb9cc12a0d1 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.344417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e5b4f-0860-4dce-8a76-ffb9cc12a0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.344683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e5b4f-0860-4dce-8a76-ffb9cc12a0d1 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.345992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 219/434] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:59.351158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e5b4f-0860-4dce-8a76-ffb9cc12a0d1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.385441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f9319-8351-4d30-b5cc-12fbc3427da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.386849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f9319-8351-4d30-b5cc-12fbc3427da6 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.397161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:59.424052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:59.463858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e5b4f-0860-4dce-8a76-ffb9cc12a0d1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:59.494601 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 216/435] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:11:59.495936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe80c8-be14-4a5b-8a12-7a271098a034 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.496230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe80c8-be14-4a5b-8a12-7a271098a034 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.497025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe80c8-be14-4a5b-8a12-7a271098a034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.499047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293b3de-8afa-4b82-8601-38790cce909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:59.500055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b3de-8afa-4b82-8601-38790cce909b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.500333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b3de-8afa-4b82-8601-38790cce909b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.500600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293b3de-8afa-4b82-8601-38790cce909b None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.502071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b3de-8afa-4b82-8601-38790cce909b None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.503536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293b3de-8afa-4b82-8601-38790cce909b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.505200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 220/436] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:59.506181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe11e32-5a3a-4331-b59e-fd4b791d9fca None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.506460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe11e32-5a3a-4331-b59e-fd4b791d9fca None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.506752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe11e32-5a3a-4331-b59e-fd4b791d9fca None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.508049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 221/437] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:11:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:11:59.508660 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe11e32-5a3a-4331-b59e-fd4b791d9fca None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.518183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-fbbe80c8-be14-4a5b-8a12-7a271098a034 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:59.519419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fbbe80c8-be14-4a5b-8a12-7a271098a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:59.584913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 217/438] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:59.585729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe80c8-be14-4a5b-8a12-7a271098a034 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.609569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe11e32-5a3a-4331-b59e-fd4b791d9fca None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:59.625807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:59.645871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4b3a2-3b55-46cd-83f9-e403f1419e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:59.646439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4b3a2-3b55-46cd-83f9-e403f1419e9d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.646737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4b3a2-3b55-46cd-83f9-e403f1419e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.647010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4b3a2-3b55-46cd-83f9-e403f1419e9d None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.648125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4b3a2-3b55-46cd-83f9-e403f1419e9d None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.649544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4b3a2-3b55-46cd-83f9-e403f1419e9d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.655892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 222/439] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:59.655892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2b60a-e25e-4044-a291-b84c7c07a501 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.655892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2b60a-e25e-4044-a291-b84c7c07a501 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.655892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2b60a-e25e-4044-a291-b84c7c07a501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.659104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:59.672040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d5a2b60a-e25e-4044-a291-b84c7c07a501 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:11:59.673223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d5a2b60a-e25e-4044-a291-b84c7c07a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:11:59.693809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe80c8-be14-4a5b-8a12-7a271098a034 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:59.728267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:59.728867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.729141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.729414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.730138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=c53f8195f07e4713b63a46a6129a59d1) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:11:59.730437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.732700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.738274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 218/440] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=c53f8195f07e4713b63a46a6129a59d1&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:59.739073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.763283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 223/441] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:11:59.850702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2b60a-e25e-4044-a291-b84c7c07a501 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.853754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cadef-c3f6-4915-81d1-106284c6f74a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:11:59.855450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:11:59.895006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:11:59.895840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:11:59.896252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:11:59.896535 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:11:59.897858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:11:59.899485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:11:59.901960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 219/442] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:11:59.907693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:11:59.986362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2b60a-e25e-4044-a291-b84c7c07a501 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.024654 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.030135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.031091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.036044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.036044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.036276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843d406-1f6f-4cf4-a304-23c09ae6177d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.043812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:00.050000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:00.051275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 224/443] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:00.052094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.076265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.076864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.077139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.077407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.078165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=dbcd6da9a237441cb483965b19c21cff, scope.project.id=a0c53322174e446daa46641a7229671f) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:00.078469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.079929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.086887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 220/444] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:11:59 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=dbcd6da9a237441cb483965b19c21cff => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:00.087495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.137576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.180825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccc476-f862-4a14-a6bc-132d9fd6364a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.234232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e424516-a817-42e5-88eb-370f80f929ad None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.240219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.240959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.241239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.241509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.255072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:00.260287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:00.261157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:00.261945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 225/445] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:00.268949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.279881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.280568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.280845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.281118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.281781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:00.282077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.283479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.292726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 221/446] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:00.299110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.332623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.391708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d74fff-dfef-4b37-bec2-e5f29daaf366 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.407628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.414767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde338be-934a-44ae-be91-19bdb62ba7b4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.456304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.456829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.457110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.457223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.457376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.457737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.458009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.458119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:00.458279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.458374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.459734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.468889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 226/447] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:00.474339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.527188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.529220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.570957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b455e-7e37-4388-b6e5-a230ea99c573 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.584562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.604637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.605230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.605500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.605787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.617898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.624189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 222/448] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:12:00.629994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.674437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.677093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.727113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763cbf6-a95a-44f2-aaab-ea51e555d3e0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.759601 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.760214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.760474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.760740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] PATH_INFO: `/v3/projects/a0c53322174e446daa46641a7229671f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.761929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] RBAC: Authorizing `identity:get_project(project_id=a0c53322174e446daa46641a7229671f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.763915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.765545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 223/449] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/projects/a0c53322174e446daa46641a7229671f => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:00.771084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.785214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 227/450] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:12:00.790793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.804426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.832853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:00.869448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb72a2-9b58-4fc5-8cca-b263ff7eb6ba None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.886564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1f6eb-90c4-4451-b109-bb3c8ff24c07 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:00.903154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.903760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.904037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.904308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] PATH_INFO: `/v3/groups/150613bd47594e6e8fec972e369754b1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.905412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] RBAC: Authorizing `identity:get_group(group_id=150613bd47594e6e8fec972e369754b1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.906864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.908411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 224/451] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/groups/150613bd47594e6e8fec972e369754b1 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:00.913987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:00.924198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:00.924812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:00.925090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:00.925361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] PATH_INFO: `/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:00.926461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] RBAC: Authorizing `identity:get_project(project_id=4cfd603abe4c44b2a5789dbc4d8a6f2b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:00.927920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:00.929791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 228/452] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/projects/4cfd603abe4c44b2a5789dbc4d8a6f2b => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:00.935290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:01.008584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d8055-fa48-499f-a807-4929b0d1f66b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:01.013266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:01.037012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:01.037996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d55e-ad02-4788-a565-996c884e658c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:01.042420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:01.043070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:01.043353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:01.043619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:01.044358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] RBAC: Adding query filter params (group.id=150613bd47594e6e8fec972e369754b1, role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=a0c53322174e446daa46641a7229671f) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:01.044665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:01.046271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:01.052574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 225/453] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/role_assignments?group.id=150613bd47594e6e8fec972e369754b1&scope.project.id=a0c53322174e446daa46641a7229671f&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:01.082475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79151498-e8f5-4f73-a84e-071f2ac092b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:01.083147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79151498-e8f5-4f73-a84e-071f2ac092b4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:01.083424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79151498-e8f5-4f73-a84e-071f2ac092b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:01.083692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79151498-e8f5-4f73-a84e-071f2ac092b4 None admin] PATH_INFO: `/v3/groups/f545c5fb848f4786aead9ee87b091c6a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:01.084953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79151498-e8f5-4f73-a84e-071f2ac092b4 None admin] RBAC: Authorizing `identity:get_group(group_id=f545c5fb848f4786aead9ee87b091c6a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:01.086430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79151498-e8f5-4f73-a84e-071f2ac092b4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:01.088106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 229/454] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 12:12:00 2022] GET /identity/v3/groups/f545c5fb848f4786aead9ee87b091c6a => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:01.094320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:01.196730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea318d3-a532-43fd-8c3d-bf27cf1c7cfc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:01.229329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333216c-d35c-46ce-bf5c-e1fe285cd027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:01.229921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333216c-d35c-46ce-bf5c-e1fe285cd027 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:01.230194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333216c-d35c-46ce-bf5c-e1fe285cd027 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:01.230461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333216c-d35c-46ce-bf5c-e1fe285cd027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:01.231736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333216c-d35c-46ce-bf5c-e1fe285cd027 None admin] RBAC: Adding query filter params (group.id=f545c5fb848f4786aead9ee87b091c6a, role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=4cfd603abe4c44b2a5789dbc4d8a6f2b) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:01.232038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333216c-d35c-46ce-bf5c-e1fe285cd027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:01.233724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333216c-d35c-46ce-bf5c-e1fe285cd027 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:01.239428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 226/455] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 12:12:01 2022] GET /identity/v3/role_assignments?group.id=f545c5fb848f4786aead9ee87b091c6a&scope.project.id=4cfd603abe4c44b2a5789dbc4d8a6f2b&role.id=753fd078fdac40abb65aa43057d79c62 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:03.533686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02fe1-d578-4f18-a23d-74001bd189d0 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:03.534588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02fe1-d578-4f18-a23d-74001bd189d0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:03.535075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02fe1-d578-4f18-a23d-74001bd189d0 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:03.536593 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 230/456] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:03.548983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390c7d-446e-4d56-9c04-a6863dad01eb None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:03.549889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390c7d-446e-4d56-9c04-a6863dad01eb None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:03.550359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390c7d-446e-4d56-9c04-a6863dad01eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:03.574931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b6390c7d-446e-4d56-9c04-a6863dad01eb None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:03.576140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b6390c7d-446e-4d56-9c04-a6863dad01eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:03.648772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:03.666431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 227/457] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:03.698494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5e31-b5e7-46c6-aa3b-6f0058716c31 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:03.698787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5e31-b5e7-46c6-aa3b-6f0058716c31 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:03.699018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5e31-b5e7-46c6-aa3b-6f0058716c31 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:03.700658 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 231/458] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:03.705895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849d59-4ed7-4fca-8638-67d17b959f84 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:03.706183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849d59-4ed7-4fca-8638-67d17b959f84 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:03.706463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f849d59-4ed7-4fca-8638-67d17b959f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:03.735275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9f849d59-4ed7-4fca-8638-67d17b959f84 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:03.736486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9f849d59-4ed7-4fca-8638-67d17b959f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:03.837218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 228/459] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:03.961876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a5e31-b5e7-46c6-aa3b-6f0058716c31 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.091251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a5e31-b5e7-46c6-aa3b-6f0058716c31 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:04.143485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:04.144116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.144395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.144666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.150675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb803-f0bc-4eac-bf7f-a746d9c178fa None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.150675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb803-f0bc-4eac-bf7f-a746d9c178fa None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.150675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb803-f0bc-4eac-bf7f-a746d9c178fa None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.153415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 229/460] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:04.157636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e7e39-6d62-406d-a086-b90a6d146235 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.158128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e7e39-6d62-406d-a086-b90a6d146235 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.158595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e7e39-6d62-406d-a086-b90a6d146235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.170258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:04.176534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:04.192076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2e9e7e39-6d62-406d-a086-b90a6d146235 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:04.193533 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2e9e7e39-6d62-406d-a086-b90a6d146235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:04.202333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:04.203282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:04.204120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:04.204120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:04.204120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:04.208269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 232/461] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:04.213838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.220313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:04.285515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 230/462] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:12:04.286689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e7e39-6d62-406d-a086-b90a6d146235 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.343076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e8a8a-b4f0-4082-94b4-7e1e731c4594 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:04.344013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:04.385459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b72640-4ffb-4799-b687-17881f27f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:04.386053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b72640-4ffb-4799-b687-17881f27f05e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.386345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b72640-4ffb-4799-b687-17881f27f05e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.387416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b72640-4ffb-4799-b687-17881f27f05e None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.387957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b72640-4ffb-4799-b687-17881f27f05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:04.388254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b72640-4ffb-4799-b687-17881f27f05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:04.390256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b72640-4ffb-4799-b687-17881f27f05e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:04.405776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 233/463] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:04.407087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e33996-c26a-4a20-a682-c7db255f1275 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.407344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e33996-c26a-4a20-a682-c7db255f1275 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.407659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e33996-c26a-4a20-a682-c7db255f1275 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.412833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 234/464] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:04.412833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e33996-c26a-4a20-a682-c7db255f1275 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.417248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e7e39-6d62-406d-a086-b90a6d146235 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:04.469185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb855da-8817-4462-8fb0-f0b4fac3c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:04.469185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb855da-8817-4462-8fb0-f0b4fac3c29a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.469185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb855da-8817-4462-8fb0-f0b4fac3c29a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.469185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb855da-8817-4462-8fb0-f0b4fac3c29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.480101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb855da-8817-4462-8fb0-f0b4fac3c29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:04.482769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb855da-8817-4462-8fb0-f0b4fac3c29a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:04.488159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-aeb855da-8817-4462-8fb0-f0b4fac3c29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:04.502734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:04.504491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:04.505413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:04.505413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:04.505413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:04.506845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 231/465] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:04.508318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01306e-b55e-4c92-a338-11e2bd8f4658 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.508808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01306e-b55e-4c92-a338-11e2bd8f4658 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.509265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01306e-b55e-4c92-a338-11e2bd8f4658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.528633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3b01306e-b55e-4c92-a338-11e2bd8f4658 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:04.530048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3b01306e-b55e-4c92-a338-11e2bd8f4658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:04.538696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e33996-c26a-4a20-a682-c7db255f1275 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:04.575508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:04.576115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.576401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.576672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.577465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:04.578917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:04.600694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 232/466] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:04.601812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01306e-b55e-4c92-a338-11e2bd8f4658 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.645602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:04.670716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 235/467] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:12:04 2022] POST /identity/v3/users => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:04.672520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.772638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01306e-b55e-4c92-a338-11e2bd8f4658 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:04.833364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:04.834386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.834677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.835118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.835949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:04.836257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:04.838416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:04.861736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bf51c-d69b-4cb1-846c-adc33960936c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:04.864318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 233/468] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:04.865649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.881327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:04.903910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:04.904607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:04.904895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:04.905171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:04.917759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:04.919799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:04.939469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:04.940465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:04.941964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:04.941964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:04.941964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:04.941964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 236/469] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:04.941964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:04.997095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:05.008751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5efee5-2fc3-4d7a-a462-6eda1aefb2cc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:05.055050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:05.056141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.056559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.057357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.058423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:05.060621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79090dd8-00fc-4870-9b11-b6d84cadba2e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:05.060856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:05.097131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:05.097756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.098032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.098300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.110082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:05.111575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:05.126673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:05.127617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:05.128494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:05.128494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:05.128494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:05.128494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 237/470] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:05.128715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:05.154954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 234/471] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:12:04 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:05.155989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:05.162470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:05.259866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec1dc-d81c-4836-9f84-afc4fa3db4b6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:05.304789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49905df-7390-40a7-9018-36076556bc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:05.305471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49905df-7390-40a7-9018-36076556bc67 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.305690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49905df-7390-40a7-9018-36076556bc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.305961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49905df-7390-40a7-9018-36076556bc67 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.306621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49905df-7390-40a7-9018-36076556bc67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:05.307244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49905df-7390-40a7-9018-36076556bc67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:05.310397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49905df-7390-40a7-9018-36076556bc67 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:05.328918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 238/472] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:05.331084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab59ed-d023-4d83-bceb-80f050077812 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.331885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab59ed-d023-4d83-bceb-80f050077812 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.332403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab59ed-d023-4d83-bceb-80f050077812 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.338965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 239/473] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:05.341478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab59ed-d023-4d83-bceb-80f050077812 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:05.356324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26782-7c45-4723-bf49-1aa6e46ae192 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:05.403460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ed59b-5172-4245-b314-3e42ced7a4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:05.403460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed59b-5172-4245-b314-3e42ced7a4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.403750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed59b-5172-4245-b314-3e42ced7a4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.403973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ed59b-5172-4245-b314-3e42ced7a4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.404612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ed59b-5172-4245-b314-3e42ced7a4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:05.404908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ed59b-5172-4245-b314-3e42ced7a4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:05.406265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ed59b-5172-4245-b314-3e42ced7a4e2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:05.420100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 235/474] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:05.421326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae4944-d2db-4b44-906f-420e2eb4d374 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.421607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae4944-d2db-4b44-906f-420e2eb4d374 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.421887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae4944-d2db-4b44-906f-420e2eb4d374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.447966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ceae4944-d2db-4b44-906f-420e2eb4d374 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:05.449557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ceae4944-d2db-4b44-906f-420e2eb4d374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:05.451980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:05.467095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:05.489493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab59ed-d023-4d83-bceb-80f050077812 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:05.529036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:05.529581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 236/475] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:05.529723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.529951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.530265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.530948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:05.531057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae4944-d2db-4b44-906f-420e2eb4d374 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:05.532507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:05.646679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae4944-d2db-4b44-906f-420e2eb4d374 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:05.657622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 240/476] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:12:05 2022] POST /identity/v3/users => generated 248 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:05.695809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:05.696435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.696761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.697107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.697778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:05.699854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:05.710414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:05.809533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:05.843851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 237/477] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:12:05 2022] POST /identity/v3/users => generated 249 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:05.938105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e121fd-c09c-4acc-97bb-9dce3ac1a46a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:05.977808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9506108-7335-4704-82bb-df756be70409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:05.978871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9506108-7335-4704-82bb-df756be70409 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:05.979402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9506108-7335-4704-82bb-df756be70409 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:05.979867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9506108-7335-4704-82bb-df756be70409 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:05.980817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9506108-7335-4704-82bb-df756be70409 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:05.982726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9506108-7335-4704-82bb-df756be70409 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:06.030209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.sql.core [None req-f9506108-7335-4704-82bb-df756be70409 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:12:06.030209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 12:12:06.030209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [parameters: {'id': '2c833dfe492740e3bd6b65f8b63b79b8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 12:12:06.030209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81680) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-f9506108-7335-4704-82bb-df756be70409 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result.read() Aug 24 12:12:06.046854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:12:06.047812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application yield resource Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise self.value Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application session.commit() Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:12:06.049258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:12:06.050372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:12:06.051525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application result.read() Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:12:06.052310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [parameters: {'id': '2c833dfe492740e3bd6b65f8b63b79b8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:06.057453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 12:12:06.060261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:06.063346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 241/478] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 12:12:05 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:06.063665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:06.180362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd955e-604d-43b8-ba7f-1d07ab5bacdd None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:06.217079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d47c5-9ba7-44f3-ac5f-2d3e82dc72b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:06.218042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d47c5-9ba7-44f3-ac5f-2d3e82dc72b4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:06.218547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d47c5-9ba7-44f3-ac5f-2d3e82dc72b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:06.219064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d47c5-9ba7-44f3-ac5f-2d3e82dc72b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:06.230250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d47c5-9ba7-44f3-ac5f-2d3e82dc72b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c77d47c5-9ba7-44f3-ac5f-2d3e82dc72b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:06.236051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:06.236945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:12:06.237729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:06.240365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 238/479] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:06.245799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9506108-7335-4704-82bb-df756be70409 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:06.254994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:06.408233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9506108-7335-4704-82bb-df756be70409 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:06.473197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038d2a3-7b6d-4cba-b4ac-75c4cfef046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:06.473754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038d2a3-7b6d-4cba-b4ac-75c4cfef046e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:06.474038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038d2a3-7b6d-4cba-b4ac-75c4cfef046e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:06.474311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038d2a3-7b6d-4cba-b4ac-75c4cfef046e None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:06.475021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038d2a3-7b6d-4cba-b4ac-75c4cfef046e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:06.475329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038d2a3-7b6d-4cba-b4ac-75c4cfef046e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:06.476720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038d2a3-7b6d-4cba-b4ac-75c4cfef046e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:06.490397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 242/480] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:06.836912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7a2da-fff9-4bc2-b60e-19900d94dfb6 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:06.837426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7a2da-fff9-4bc2-b60e-19900d94dfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:06.837426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7a2da-fff9-4bc2-b60e-19900d94dfb6 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:06.839079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 239/481] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:06.845289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89418a77-7f5f-4015-8403-3efe6537647e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:06.845779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89418a77-7f5f-4015-8403-3efe6537647e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:06.845853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89418a77-7f5f-4015-8403-3efe6537647e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:06.871360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-89418a77-7f5f-4015-8403-3efe6537647e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:06.872459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-89418a77-7f5f-4015-8403-3efe6537647e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:06.938457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:06.949575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 243/482] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:07.300949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7a2da-fff9-4bc2-b60e-19900d94dfb6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:07.371427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:07.448315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7a2da-fff9-4bc2-b60e-19900d94dfb6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:07.456034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12af51-e29c-4772-bc96-fa0b860452ea None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:07.456322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12af51-e29c-4772-bc96-fa0b860452ea None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:07.456599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12af51-e29c-4772-bc96-fa0b860452ea None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:07.458193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 244/483] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:07.488379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf284e8-f0ed-43f5-813a-e0ed572f4a45 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:07.488666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf284e8-f0ed-43f5-813a-e0ed572f4a45 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:07.488946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf284e8-f0ed-43f5-813a-e0ed572f4a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:07.491553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:07.492166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:07.492440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:07.492709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:07.503095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:07.508445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:07.509392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:07.510175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:07.514772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 240/484] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:07.517131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:07.537473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1bf284e8-f0ed-43f5-813a-e0ed572f4a45 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:07.538701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1bf284e8-f0ed-43f5-813a-e0ed572f4a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:07.652316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57986d1-1c1b-471f-ac43-76c908eaeac8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:07.683010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:07.704117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 245/485] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:07.720172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:07.720791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:07.721071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:07.721345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:07.722016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:07.722319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:07.723732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:07.736430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 241/486] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:07.741945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf284e8-f0ed-43f5-813a-e0ed572f4a45 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:07.853128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf284e8-f0ed-43f5-813a-e0ed572f4a45 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:07.878018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:07.889350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:07.890022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:07.890298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:07.890567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:07.900845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:07.902703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:07.917713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:07.919328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:07.920182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:07.920182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:07.920182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:07.924191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 246/487] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:07.925572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.028287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dcb66-edc2-476a-b8f9-dd13dca02748 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:08.063354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:08.063960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.064242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.064509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.065151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:08.065445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:08.066892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:08.082783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 242/488] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:08.084793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.198642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd1d63-b89a-468c-b1d2-8cfa171d0300 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:08.233583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcaaf1c-13f2-48a6-af51-8081b7dd6b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:08.234224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcaaf1c-13f2-48a6-af51-8081b7dd6b67 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.234499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcaaf1c-13f2-48a6-af51-8081b7dd6b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.234805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcaaf1c-13f2-48a6-af51-8081b7dd6b67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.237517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.247906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcaaf1c-13f2-48a6-af51-8081b7dd6b67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:08.249367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcaaf1c-13f2-48a6-af51-8081b7dd6b67 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-6bcaaf1c-13f2-48a6-af51-8081b7dd6b67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:08.262158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:08.263113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:08.264042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 247/489] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:12:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:08.265122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f3796-be85-44f9-b8a4-80fde8d6efb9 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.265122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f3796-be85-44f9-b8a4-80fde8d6efb9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.265273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f3796-be85-44f9-b8a4-80fde8d6efb9 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.266771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 248/490] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:08.272171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55257b0d-7ccc-4a73-a78c-8e2101ad24b0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.272563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55257b0d-7ccc-4a73-a78c-8e2101ad24b0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.272942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55257b0d-7ccc-4a73-a78c-8e2101ad24b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.282442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:08.292492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-55257b0d-7ccc-4a73-a78c-8e2101ad24b0 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:08.309326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-55257b0d-7ccc-4a73-a78c-8e2101ad24b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:08.327903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:08.363761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242372-3b0d-4c27-8b99-a4150612088e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:08.367946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 249/491] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:08.369062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55257b0d-7ccc-4a73-a78c-8e2101ad24b0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.409285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd5680-eb40-49b4-90e7-09d9bec99b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:08.410821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd5680-eb40-49b4-90e7-09d9bec99b23 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.411107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd5680-eb40-49b4-90e7-09d9bec99b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.417147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd5680-eb40-49b4-90e7-09d9bec99b23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.436964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd5680-eb40-49b4-90e7-09d9bec99b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-11fd5680-eb40-49b4-90e7-09d9bec99b23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:08.442999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:08.444738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:08.445894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:08.449260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 243/492] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:08.451269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834c697-d5fd-4f97-9656-acbdc381c32d None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.451561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834c697-d5fd-4f97-9656-acbdc381c32d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.451837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834c697-d5fd-4f97-9656-acbdc381c32d None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.453393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 244/493] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:08.454108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834c697-d5fd-4f97-9656-acbdc381c32d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.510092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55257b0d-7ccc-4a73-a78c-8e2101ad24b0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:08.555432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53911a-a436-41cb-9117-bf4c6fac8b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:08.556612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53911a-a436-41cb-9117-bf4c6fac8b0c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.557081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53911a-a436-41cb-9117-bf4c6fac8b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.557669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53911a-a436-41cb-9117-bf4c6fac8b0c None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.558304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:08.559057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53911a-a436-41cb-9117-bf4c6fac8b0c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:08.559827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53911a-a436-41cb-9117-bf4c6fac8b0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:08.561870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53911a-a436-41cb-9117-bf4c6fac8b0c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:08.566349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834c697-d5fd-4f97-9656-acbdc381c32d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:08.604898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:08.606470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.607130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.607946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.608789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:08.609294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:08.610884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:08.614567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 250/494] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:12:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:08.616298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86cd75-a229-42e6-9c03-b0258b484e9a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.616589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86cd75-a229-42e6-9c03-b0258b484e9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.616868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86cd75-a229-42e6-9c03-b0258b484e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.623703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 245/495] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:08.625840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.643753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fe86cd75-a229-42e6-9c03-b0258b484e9a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:08.644972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fe86cd75-a229-42e6-9c03-b0258b484e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:08.667035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:08.743020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 251/496] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:08.744519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86cd75-a229-42e6-9c03-b0258b484e9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.761305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398c432-e33b-4755-9d0a-d6b7a320e0e5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:08.808014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:08.809049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.810104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.810593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.855822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:08.870868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86cd75-a229-42e6-9c03-b0258b484e9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:08.912132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:08.913686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:08.916225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:08.917299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:08.918543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:08.919177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:08.936522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:08.939783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:08.959578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:08.960553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:08.961387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:08.961387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:08.961387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:08.963503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 252/497] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:08.964579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:08.981279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.062893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 246/498] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:09.063695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.082637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0d366-c449-4a36-8ca2-851675826308 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.117245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.117855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.118127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.118393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.121815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.128179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:09.133356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:09.134345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:09.135177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 253/499] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:09.136793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.162318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986aa94-bb57-4a95-84ad-e4c605bbb449 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.195681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.196270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.196549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.196819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.197453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:09.197756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.199181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.199851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.210193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 247/500] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:09.211015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.234395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c7f29-a226-4ce6-914f-687b3ce6fea9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.268240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558903b-bb44-4e33-9f5a-de43d4374d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.268852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558903b-bb44-4e33-9f5a-de43d4374d0e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.269126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558903b-bb44-4e33-9f5a-de43d4374d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.269389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558903b-bb44-4e33-9f5a-de43d4374d0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.279911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558903b-bb44-4e33-9f5a-de43d4374d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3558903b-bb44-4e33-9f5a-de43d4374d0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:09.286362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:09.287268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:09.288409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 254/501] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:09.289283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5b109-9cc6-4d52-8701-f056ec169b9a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.289283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5b109-9cc6-4d52-8701-f056ec169b9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.289283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5b109-9cc6-4d52-8701-f056ec169b9a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.290463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 255/502] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:09.291029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5b109-9cc6-4d52-8701-f056ec169b9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.314476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30d227-1a6a-4d9e-945c-183b6920316f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.330934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.348409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.349022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.349304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.349578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.360300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.361999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:09.377160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:09.378049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:09.379259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:09.379259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:09.379259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:09.379259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 248/503] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:09.379259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.397528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5b109-9cc6-4d52-8701-f056ec169b9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.422567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.434519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.435147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.435422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.435686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.436333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:09.436628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.438001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.446517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 256/504] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:09.447481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.489305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1843796-1275-41ed-8775-8cdd49ddfd66 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.521750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7793ff-2b88-4a9d-99c4-2ee6bdb3cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.522370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7793ff-2b88-4a9d-99c4-2ee6bdb3cb1d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.522668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7793ff-2b88-4a9d-99c4-2ee6bdb3cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.522937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7793ff-2b88-4a9d-99c4-2ee6bdb3cb1d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.530202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7793ff-2b88-4a9d-99c4-2ee6bdb3cb1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.531779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7793ff-2b88-4a9d-99c4-2ee6bdb3cb1d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-7a7793ff-2b88-4a9d-99c4-2ee6bdb3cb1d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:09.544325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:09.545209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:09.546083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 249/505] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:09.546625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7adfb-5758-4594-8df9-325bb297903b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.546733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95938b-3955-4935-9a5f-39cd938dc390 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.546803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95938b-3955-4935-9a5f-39cd938dc390 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.547335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95938b-3955-4935-9a5f-39cd938dc390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.564771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cf95938b-3955-4935-9a5f-39cd938dc390 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:09.565903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cf95938b-3955-4935-9a5f-39cd938dc390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:09.584369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.584961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.585235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.585498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.586143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:09.586439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.587825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.589697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.595499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 257/506] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:09.596269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.632771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 250/507] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:09.633603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95938b-3955-4935-9a5f-39cd938dc390 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.649472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.701670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21793fb-1766-4d3d-a9a9-0ddf180c1c47 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.737579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95938b-3955-4935-9a5f-39cd938dc390 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.737739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.738234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.738508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.738801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.739432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:09.739724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.741083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.757607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 258/508] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:09.757607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.777577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.778203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.778457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.778754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.789423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.790869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:09.806972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:09.808171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:09.809833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:09.809833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:09.809833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:09.813482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 251/509] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:09.814356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.829453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.863742 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877d5ff-5aa9-4add-8da8-019ec150475a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.876655 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:09.906784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.907705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.907989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.908425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.909196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:09.909505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.911272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.928931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044703f9-44da-4481-a1cc-ce8c81e45ce2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:09.929896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 259/510] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:09.930727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:09.967547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:09.968168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:09.968449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:09.968723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:09.979065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:09.980550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:09.994296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:09.995434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:09.996765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:09.996765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:09.996765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:09.996765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 252/511] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:09.996765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.033933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4f733-89f4-4616-8def-baa5d639d716 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.057214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.067782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.069477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.069477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.069477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.075341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.076837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:10.088146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:10.088994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:10.090347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 260/512] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:10.090851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.099938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecd54-5a0e-4801-a9fa-736bf2e86225 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.118102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.133236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.133843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.134122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.134395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.135070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:10.135374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.137535 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.150774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 253/513] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:10.151609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.199533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b61790-babd-4441-a2c6-0aa7798ea2a0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.233580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.234196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.234469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.234770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.244463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.245930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.258634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69d59d-8a83-4bac-b8b8-9d9262bf7cc7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:10.260402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:10.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:10.262430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:10.262430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:10.262430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:10.262430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 261/514] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:10.262430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.283114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.293694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.294300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.294552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.294840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.340676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.360075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb50658-cf90-4b2b-90d0-989d3ec3f573 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.361162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.362685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.394061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.394662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.394955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.395229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.395854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:10.396160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.397516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.408888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 262/515] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:10.409667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.466998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 254/516] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:10.467847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.494525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.512308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84694-0f5b-4406-aba1-a10de43da218 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.550986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.551574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.551848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.552117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.552795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:10.553095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.554518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.568622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50540f2-7130-4917-b654-921187a792c5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.570299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.573537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 263/517] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:10.574535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.604196 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.604741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.605019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.605916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.613303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.614777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:10.625991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:10.626927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 255/518] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:10.628155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.675294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2187727-219b-4836-b1d6-d955a855549c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.708398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.709080 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.709313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.709430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.709623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.710322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:10.710563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.711951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.723469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 264/519] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:10.724243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.725211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47f596-91d2-47e1-8f55-f8fba28db5a7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.759266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.760034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.760377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.760646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.767959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.769404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:10.780523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:10.781409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 256/520] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:10.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.789990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:10.830343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c85b1-8849-4ccc-b5ca-93a766a233bf None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.865671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.866142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.866419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.866715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.877202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.878731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.886883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40135805-2e4d-4235-ab50-3976cdb1088e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:10.892823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:10.893737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:10.894879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:10.894879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:10.894879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:10.894879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 265/521] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:10.895394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.924103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:10.924686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:10.924959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:10.925223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:10.925938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=0ed16fea638a45938eee8cdf234333f1) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:10.926248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:10.927706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:10.932986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 257/522] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/role_assignments?user.id=0ed16fea638a45938eee8cdf234333f1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:10.933783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:10.948237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.003057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ee43a-d032-4645-83b5-a167584255ac None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.018097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.041532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.042146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.042422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.042739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.043463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:11.043766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.045341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.061880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad580bef-84f2-4d86-87c0-0273b2b33c1c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.070811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 266/523] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 12:12:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:11.070811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.100010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.101040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.101331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.101605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.102527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.104461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.177571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f1c03-04fe-4b9f-bd49-d3f580ff9f76 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.194692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 258/524] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:12:10 2022] POST /identity/v3/users => generated 247 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:11.194692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.230289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.231138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.231517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.231889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.233001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:11.233402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.234982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.254782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 267/525] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:11.255915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.268390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.287407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.315345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdbc01-d79d-42aa-ac7c-4f6a03d71a48 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.354531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.355354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.355640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.355917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.356578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:11.356886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.358858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.365853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653c33f-d9a1-428f-bd5f-4f736c0f8501 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.373660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 259/526] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:11.374536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.406636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.407268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.407547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.407816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.497709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.499599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.500049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece94b8-ba9e-4fba-85e0-9b9c3ad60bc1 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.525276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.532169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.536460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.537130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.537471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.542803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.605997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 268/527] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:11.606948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.609447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.611069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.709415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ee795-88fe-4f20-98fc-2fcf714dcbd9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.719037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 260/528] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:11.720101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.737184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.744914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.751200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.751200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.751200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.753695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.755183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:11.766482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:11.767499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:11.768364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 269/529] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:11.768827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.774234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:11.839526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e91a7-0037-4e50-881d-a08782535711 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.877464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a5432-0800-48ed-8e40-efd0047955b0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:11.878106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.878780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.879287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.879560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.893572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.895568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.914167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:11.914353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:11.915437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:11.916440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:11.916440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:11.916440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:11.916440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 261/530] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:11.916722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:11.918929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:11.918929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:11.918929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:11.932995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:11.935165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:11.950434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:11.951581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:11.952619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:11.952619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:11.952619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:11.955035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 270/531] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:11.955035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.003481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:12.005350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:12.028464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5e675-84ae-4858-9f02-7336678464cd None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.065277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b456de1-96aa-4b06-b005-4844cdc22a3a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.067183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.070812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.070812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.070812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.070812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:12.070812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.071523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.095228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 262/532] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:12.096309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.100764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.101366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.101639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.101906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.102541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:12.102870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.104227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.117801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 271/533] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:12.118631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.204540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f120df-6346-4f32-88c5-1086f7e144d6 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.218904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:12.221963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50fd43-e323-493e-888c-f68a4aefa972 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.243948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.244555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.244836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.245115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.245783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:12.246086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.247537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.250042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:12.258391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.259096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.259699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.260049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.260620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 263/534] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:12.260899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=2f33e1c573c044f086f99eb128c8f726) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:12.261271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.261473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.262966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.269210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 272/535] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/role_assignments?user.id=2f33e1c573c044f086f99eb128c8f726&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:12.274774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.384416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c294476-970f-4ac0-9fbb-2a178a79ad34 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.459852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.460501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.460781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.461054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.475035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a165-baf3-44ac-8427-92c62969eabe None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.479903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.481405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:12.496003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:12.496940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:12.497863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:12.498847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 273/536] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:12.504613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.514582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.515295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.515580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.515851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.524253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.525722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:12.538897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:12.540979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:12.543163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 264/537] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:12.551779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.566697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:12.659940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445ff46-ad5c-4c7b-8edb-8dccd403f402 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.697600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.698191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.698473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.698768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.700213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:12.700524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.702077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.716221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 274/538] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:12.721960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:12.735701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:12.820871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:12.839154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73ace4-f35b-412d-855d-f184e6977229 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.852854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d522d4-91df-49af-9f95-e80714f9c3bc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:12.881656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd3263-bd78-4371-b0c8-816d4577cf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.882325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd3263-bd78-4371-b0c8-816d4577cf7c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.882539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd3263-bd78-4371-b0c8-816d4577cf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.882833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd3263-bd78-4371-b0c8-816d4577cf7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.883722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd3263-bd78-4371-b0c8-816d4577cf7c None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=639c0cb750744031b47ba3e48170cce1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:12.884039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd3263-bd78-4371-b0c8-816d4577cf7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.885473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd3263-bd78-4371-b0c8-816d4577cf7c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.892469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 275/539] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/role_assignments?user.id=639c0cb750744031b47ba3e48170cce1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:12.934622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04def81-2466-425d-8dbc-9dd58c40c135 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.935274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04def81-2466-425d-8dbc-9dd58c40c135 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.935746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04def81-2466-425d-8dbc-9dd58c40c135 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.938734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 276/540] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:12:12.943604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f815b4-0232-4c56-97b6-c785521b57bc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.944204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f815b4-0232-4c56-97b6-c785521b57bc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.944699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f815b4-0232-4c56-97b6-c785521b57bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.964040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c0f815b4-0232-4c56-97b6-c785521b57bc None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:12.966204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c0f815b4-0232-4c56-97b6-c785521b57bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:12.971414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:12.972049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:12.972329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:12.972603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:12.973322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:12.973629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:12.975174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:12.991975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 265/541] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:13.002808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:13.098764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 277/542] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:12:13.136592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629fc86-0b91-4e50-920f-6c98aca93e87 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:13.155592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:13.173702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58708634-209f-486e-8b15-97ef0828f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:13.174483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708634-209f-486e-8b15-97ef0828f01b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:13.178796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708634-209f-486e-8b15-97ef0828f01b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:13.178796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58708634-209f-486e-8b15-97ef0828f01b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:13.178796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58708634-209f-486e-8b15-97ef0828f01b None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=422f81200e5645668e123229a6f356d3) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:13.178796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58708634-209f-486e-8b15-97ef0828f01b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:13.178796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58708634-209f-486e-8b15-97ef0828f01b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:13.184107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 266/543] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:12 2022] GET /identity/v3/role_assignments?user.id=422f81200e5645668e123229a6f356d3&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:13.370525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8629-85a4-4ff0-8b2a-e938d76454ed None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:13.370918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8629-85a4-4ff0-8b2a-e938d76454ed None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:13.371092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8629-85a4-4ff0-8b2a-e938d76454ed None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:13.372523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 278/544] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:13.377878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879698c5-89a8-49dc-8574-a617690075cf None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:13.378307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879698c5-89a8-49dc-8574-a617690075cf None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:13.378493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879698c5-89a8-49dc-8574-a617690075cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:13.401158 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-879698c5-89a8-49dc-8574-a617690075cf None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:13.403231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-879698c5-89a8-49dc-8574-a617690075cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:13.469675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8629-85a4-4ff0-8b2a-e938d76454ed None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:13.489592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 267/545] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:13.560873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:13.585632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8629-85a4-4ff0-8b2a-e938d76454ed None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:13.623167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:13.624155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:13.624634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:13.625113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:13.635805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:13.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:13.642129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:13.642978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:13.649963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 279/546] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:13.651580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879698c5-89a8-49dc-8574-a617690075cf None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:13.756208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879698c5-89a8-49dc-8574-a617690075cf None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:13.767502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:13.791819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:13.793353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:13.793832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:13.794301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:13.795227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:13.795804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:13.797361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:13.810455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 268/547] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:13.812067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:13.919388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7706813-fba8-447f-8a53-d4481d178204 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:13.954966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:13.956783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:13.967446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:13.967996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:13.968471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:13.982015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:13.991770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:13.993147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:14.016257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:14.017298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:14.018791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:14.018791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:14.018791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:14.018791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 280/548] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:14.023018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:14.115092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab06c46-d7e2-493b-8e1d-aee9b2ac9294 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:14.165671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:14.166683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.167193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.168172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.182851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:14.189835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:14.190964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:14.191786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:14.194692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 269/549] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:14.204611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:14.261831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd304d27-d458-4933-a3dc-795a185082f0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:14.275001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:14.331865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:14.332471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.332752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.333027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.333671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:14.334604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:14.336236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:14.362632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 281/550] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:14.366735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:14.406416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796647dd-d42d-4f1d-b793-239d4ebd7208 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:14.464712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:14.474053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:14.474747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.475083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.482945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.482945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:14.482945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:14.482945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:14.499372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 270/551] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:14.507154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:14.574637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8dc4-596b-441a-8596-bf0361fa84e9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:14.610932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:14.618001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5092df1-0cb6-4803-a033-9ea7ef447a72 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:14.625502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d373af-fba5-4df5-a4c1-02f0b020444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:14.627685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d373af-fba5-4df5-a4c1-02f0b020444f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.627685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d373af-fba5-4df5-a4c1-02f0b020444f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.627685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d373af-fba5-4df5-a4c1-02f0b020444f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.637422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d373af-fba5-4df5-a4c1-02f0b020444f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:14.638928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d373af-fba5-4df5-a4c1-02f0b020444f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:14.654602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f41f4-e703-463c-80b6-3b28f97faf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e8d373af-fba5-4df5-a4c1-02f0b020444f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:14.654891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:14.655777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:14.657185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:14.657723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f41f4-e703-463c-80b6-3b28f97faf09 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.657723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f41f4-e703-463c-80b6-3b28f97faf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.657723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f41f4-e703-463c-80b6-3b28f97faf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.657909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 282/552] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:12:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:14.657909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7931-5b36-416d-b081-07e61067cb2e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.657909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7931-5b36-416d-b081-07e61067cb2e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.658081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b7931-5b36-416d-b081-07e61067cb2e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.664035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 283/553] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:14.665390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b7931-5b36-416d-b081-07e61067cb2e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:14.671320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f41f4-e703-463c-80b6-3b28f97faf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:14.672764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f41f4-e703-463c-80b6-3b28f97faf09 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-d50f41f4-e703-463c-80b6-3b28f97faf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:14.691348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:14.693687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:14.695910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:14.695910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:14.695910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:14.695910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 271/554] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:14.695910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586695e-0450-4f74-af09-ab0ef32a62d4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.695910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586695e-0450-4f74-af09-ab0ef32a62d4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.695910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586695e-0450-4f74-af09-ab0ef32a62d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.719807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d586695e-0450-4f74-af09-ab0ef32a62d4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:14.721460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d586695e-0450-4f74-af09-ab0ef32a62d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:14.778172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b7931-5b36-416d-b081-07e61067cb2e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:14.790357 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:14.808825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 272/555] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:14.809859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586695e-0450-4f74-af09-ab0ef32a62d4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:14.816219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:14.816930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.817306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.817723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.818725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:14.819152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:14.820857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:14.841448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 284/556] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:12:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:14.846710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:14.901132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:14.926101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586695e-0450-4f74-af09-ab0ef32a62d4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:14.970168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:14.970705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:14.971660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:14.973216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:14.973806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:14.974965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:14.976284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:14.980808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd696e-a609-4af7-a9e9-774cc49bd821 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.000794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 273/557] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:15.006997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.040756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.041373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.041647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.041962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.059930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.063180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:15.087536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:15.090874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:15.093032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:15.093032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:15.093032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:15.093032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 285/558] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:15.112283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.162529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.165781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f7225-0c19-424a-a345-2f7d70877281 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.219610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.220435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.220659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.221019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.223997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.229695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.231311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:15.244241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:15.247135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 274/559] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:15.249446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.270149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9a871-88ae-4a29-8d08-0891fc6f7f5d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.303863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.304870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.305162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.305437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.306235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:15.306558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.307958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.325845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 286/560] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:15.325845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.359521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15e1c5-33f4-43b3-9b0e-4f94a0f0a81d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.398239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb3d07-4ee9-46db-bf80-d6bb3b998582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.398928 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb3d07-4ee9-46db-bf80-d6bb3b998582 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.399212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb3d07-4ee9-46db-bf80-d6bb3b998582 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.399486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb3d07-4ee9-46db-bf80-d6bb3b998582 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.408436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.415243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb3d07-4ee9-46db-bf80-d6bb3b998582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-10fb3d07-4ee9-46db-bf80-d6bb3b998582 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:15.420894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:15.422593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 275/561] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:15.423469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cd82a-c88b-4ecf-bca2-d384653d0193 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.424292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cd82a-c88b-4ecf-bca2-d384653d0193 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.424292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cd82a-c88b-4ecf-bca2-d384653d0193 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.432241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 276/562] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:15.432241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cd82a-c88b-4ecf-bca2-d384653d0193 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.444190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3918d-04e8-4b8b-b9de-658bb5fd2977 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.476930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.477443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.477721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.477989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.478736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:15.479039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.481099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.486512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.498958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 287/563] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:15.501197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.531202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cd82a-c88b-4ecf-bca2-d384653d0193 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.565270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d8c6e-5530-4085-a3af-ff5f57e3d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.565973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d8c6e-5530-4085-a3af-ff5f57e3d93c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.566326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d8c6e-5530-4085-a3af-ff5f57e3d93c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.566691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d8c6e-5530-4085-a3af-ff5f57e3d93c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.573104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d8c6e-5530-4085-a3af-ff5f57e3d93c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.574558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d8c6e-5530-4085-a3af-ff5f57e3d93c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-164d8c6e-5530-4085-a3af-ff5f57e3d93c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:15.585476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:15.586340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:15.587273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 277/564] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:15.588104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b1627-8cef-4a87-992a-11c24e08c993 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.588392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b1627-8cef-4a87-992a-11c24e08c993 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.588666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b1627-8cef-4a87-992a-11c24e08c993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.604670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d79224-7139-4b8e-b7af-8f36229a0550 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.606368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f67b1627-8cef-4a87-992a-11c24e08c993 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:15.607874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f67b1627-8cef-4a87-992a-11c24e08c993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:15.642403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c9593-dea7-4418-b1c8-647ce95e8e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.643026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9593-dea7-4418-b1c8-647ce95e8e65 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.643302 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9593-dea7-4418-b1c8-647ce95e8e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.643567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9593-dea7-4418-b1c8-647ce95e8e65 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.644236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c9593-dea7-4418-b1c8-647ce95e8e65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:15.644532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c9593-dea7-4418-b1c8-647ce95e8e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.645873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c9593-dea7-4418-b1c8-647ce95e8e65 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.650776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.654030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 288/565] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:15.655188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd480cd6-7b9c-46fd-821d-1f651cd97791 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.655455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd480cd6-7b9c-46fd-821d-1f651cd97791 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.655729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd480cd6-7b9c-46fd-821d-1f651cd97791 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.657045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 289/566] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:15.657627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd480cd6-7b9c-46fd-821d-1f651cd97791 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.676630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 278/567] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:15.677463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b1627-8cef-4a87-992a-11c24e08c993 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.735133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.766326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd480cd6-7b9c-46fd-821d-1f651cd97791 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.789805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b1627-8cef-4a87-992a-11c24e08c993 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.804663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bf710-13ae-4755-9a02-49c29be49dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.805275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bf710-13ae-4755-9a02-49c29be49dca None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.805556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bf710-13ae-4755-9a02-49c29be49dca None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.805834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bf710-13ae-4755-9a02-49c29be49dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.827516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.828300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.828564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.828832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.829538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:15.829827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.831302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.849705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 279/568] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:15.849705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:15.883493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.886036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bf710-13ae-4755-9a02-49c29be49dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:15.887532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bf710-13ae-4755-9a02-49c29be49dca None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:15.953906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb215af9-037f-4abc-a255-2f91c70dd81a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:15.960685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:15.987446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:15.988001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.988311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.993079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:15.993206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 290/569] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:15.993206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62866737-b425-4c57-9bde-e6e4ff2dfdd1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:15.993206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62866737-b425-4c57-9bde-e6e4ff2dfdd1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:15.993206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62866737-b425-4c57-9bde-e6e4ff2dfdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.000961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.002413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.007266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-62866737-b425-4c57-9bde-e6e4ff2dfdd1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:16.008436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-62866737-b425-4c57-9bde-e6e4ff2dfdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:16.016146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:16.017205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:16.018106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:16.018106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:16.018106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:16.018106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 280/570] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:16.018407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.076652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 291/571] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:16.077477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62866737-b425-4c57-9bde-e6e4ff2dfdd1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.107611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:16.119249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231690bf-8e3c-43e2-a456-93cfba0094b6 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.155233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.156208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.156682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.157172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/0ed16fea638a45938eee8cdf234333f1/roles/c26057e0945745a39399a115b756ea2b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.173144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=0ed16fea638a45938eee8cdf234333f1, role_id=c26057e0945745a39399a115b756ea2b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.174581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.179856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62866737-b425-4c57-9bde-e6e4ff2dfdd1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.213353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.213966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.214238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.215072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.224775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:16.229387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:16.230459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:16.231345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 292/572] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:16.232422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.259056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 281/573] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:16 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/0ed16fea638a45938eee8cdf234333f1/roles/c26057e0945745a39399a115b756ea2b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:16.259056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.270815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:16.359957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:16.370710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001a783-6681-4e87-8642-3f468de031f3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.375376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961266de-1db2-4deb-a959-4a17c63643a7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.410072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.410747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.411109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.413049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.415539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.415539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.416072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.416072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.416617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:16.417646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.418324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.429344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.430939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.433602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 282/574] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:16.434173 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:16.446909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:16.447903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:16.449986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:16.449986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:16.449986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:16.449986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 293/575] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:16.449986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.506106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:16.540814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b21ba-f51c-4a8b-ae5c-472d30da1da5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.571372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fb584-bd8a-42f8-9e7a-dc5561bed3c1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.573508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.574695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.574977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.575248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.575917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:16.576218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.577591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.586087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 283/576] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:16.586911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.613378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.614291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.614785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.615323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.626693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:16.632454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:16.633378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:16.634213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:16.641290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 294/577] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:16.642323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.645405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:16.689694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bb933-9ada-4578-bb78-fe052c54bb8b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.725446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:16.731405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.732028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.732568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.732832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.740423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.741843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.747143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0054b6-9847-4a53-8252-bbac6603be90 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:16.753425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:16.754400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:12:16.755912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:16.759508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 284/578] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:16.760684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.784092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.784937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.785053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.785237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.785875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:16.786180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.787587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.800470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 295/579] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:16.801212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.865020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:16.883018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b7fc-65d4-4095-9a11-4f259ae89536 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.906667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c332e5-5921-4b0f-9b91-bce32af0c1db None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:16.918556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.919205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.919481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.919747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.930628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.932119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.942198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:16.942835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:16.943118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:16.943391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:16.944052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:16.944356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:16.945710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:16.946627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:16.947582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:16.948583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:16.948583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:16.948583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:16.948583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 285/580] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:16.948583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.959562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 296/581] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:16.960298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:16.995136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.050429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6e617-c8f7-4a8f-bc9d-574a9204f8dd None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.069702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13777cd5-dfa0-43e7-83d8-7bb3a7fe08ae None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.084456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.085056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.085335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.085605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.086326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:17.086630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.088087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.094801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.107090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.108205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.108684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.109124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.116136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.117837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.125312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 286/582] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:17.126210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:17.132249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:17.133158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:17.134047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:17.136329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 297/583] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:17.137387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.237640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.240913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddf404-73d4-4157-b627-0a104011992d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.247165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfc149-e6e7-45e1-93f9-4a5c79ee4344 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.278885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.279487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.279767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.280035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.280673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:17.281030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.281640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.282253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.282348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.282528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.282856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.294238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.295373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 287/584] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:17.296193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.296427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:17.313093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:17.314286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:17.315415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:17.315415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:17.315415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:17.320837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 298/585] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:17.320837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.358268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.420833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb8e1b-1cc1-4d01-98d0-916b8f54c429 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.436552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4813b85-83a1-4a50-b164-611119c5507a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.460773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.461486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.461827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.462181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.473935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.475421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.477584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.478437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.478892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.479303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.480737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:17.481185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.483171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:17.491132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:17.492200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:17.493226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:17.493226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:17.493226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:17.499457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 288/586] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:17.499457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.500661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 299/587] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:17.500993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.512004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.598228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.603534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f05ba-36eb-42f5-a133-594111047298 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.603937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a77d9-395c-44f7-870a-b2aab188c910 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.639813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.640651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.640876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.641244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.641497 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.641769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.641978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.642083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.642868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:17.643328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.644852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.649252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.650680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.657573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 300/588] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:17.658191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:17.666940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:17.668462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 289/589] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:17.669410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.747441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.762543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580f9ef-4ff1-4df8-a2a9-2c508315bc26 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.778172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765d2d-0b87-47e4-b929-e58854c09d84 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.798351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.799104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.799450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.799799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.800506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:17.800875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.802369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.813812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:17.818771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:17.818920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 301/590] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:17.819015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:17.819015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:17.819179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.819264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=909708a2a44a48e7b62a6f6a8d792f11) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:17.819264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:17.819264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:17.829210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 290/591] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/role_assignments?user.id=909708a2a44a48e7b62a6f6a8d792f11&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:17.829210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:17.848328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:17.960149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84a710-95a5-42b0-bf63-029a70f4d1bc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:17.960747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bdb7d-644a-4acf-8927-34c79c7c779d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.003019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.003909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.004279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.004600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.005224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.005976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.006450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.006893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.007757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:18.008261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.009884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.014129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.015012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.016544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:18.030855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:18.031838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:12:18.032759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:18.038100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 302/592] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:18.038100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.038757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 291/593] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 24 12:12:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:18.039867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.144293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.150523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e7734-a296-46d9-8470-46d3aaea6718 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.173174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bded-7950-4309-8b11-13aff5210d29 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.189627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.190603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.191176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.191746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.203578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.205314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.216564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71845b-ed2b-4cd8-a987-4a59d02314ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.217190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71845b-ed2b-4cd8-a987-4a59d02314ca None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.217472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71845b-ed2b-4cd8-a987-4a59d02314ca None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.217746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71845b-ed2b-4cd8-a987-4a59d02314ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:18.222632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:18.223574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:18.224485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:18.224485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:18.224485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:18.225242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71845b-ed2b-4cd8-a987-4a59d02314ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.226427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 292/594] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:18.226772 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71845b-ed2b-4cd8-a987-4a59d02314ca None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.227137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2e71845b-ed2b-4cd8-a987-4a59d02314ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:18.242371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:18.243368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:18.244248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 303/595] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:18.244741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf451032-23fd-4ab7-9ca1-cf3cc5f2c18b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.245981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf451032-23fd-4ab7-9ca1-cf3cc5f2c18b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.245981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf451032-23fd-4ab7-9ca1-cf3cc5f2c18b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.247249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 304/596] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:18.249189 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf451032-23fd-4ab7-9ca1-cf3cc5f2c18b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.284230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.342012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70edbf-1f3d-4fd4-aad5-7d2efa4d4596 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.362162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf451032-23fd-4ab7-9ca1-cf3cc5f2c18b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.378286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.378945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.379220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.379520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.380375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:18.380681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.383136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.400893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b93bb-f222-42e8-a515-2058da8a6056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.401415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b93bb-f222-42e8-a515-2058da8a6056 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.401723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b93bb-f222-42e8-a515-2058da8a6056 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.401995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b93bb-f222-42e8-a515-2058da8a6056 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.402634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b93bb-f222-42e8-a515-2058da8a6056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:18.402974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b93bb-f222-42e8-a515-2058da8a6056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.409556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b93bb-f222-42e8-a515-2058da8a6056 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.409746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 293/597] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:18.409746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.411797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.432372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 305/598] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:18.433635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93e0b-51a0-4154-a53d-cbefa386cf8d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.433969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93e0b-51a0-4154-a53d-cbefa386cf8d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.434243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c93e0b-51a0-4154-a53d-cbefa386cf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.456922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c9c93e0b-51a0-4154-a53d-cbefa386cf8d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:18.458130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c9c93e0b-51a0-4154-a53d-cbefa386cf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:18.527663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216528f-d150-49f8-a98a-e2752ab5f8ff None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.530584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 306/599] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:18.530893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c93e0b-51a0-4154-a53d-cbefa386cf8d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.531384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.587296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.588652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.589833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.590300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.591739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:18.592252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.594417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.610482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 294/600] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:18.611628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.646261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c93e0b-51a0-4154-a53d-cbefa386cf8d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.673316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.685613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.686360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.687256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.687256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.697723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.699738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:18.716444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:18.717475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:18.718384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:18.718384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:18.718384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:18.718384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 307/601] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:18.718384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.742720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e8921-265d-43f9-865b-4ab68c6b5240 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.787529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.788629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.789113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.789675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.797201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.800409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:18.814448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:18.815398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:18.816285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:18.820728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 295/602] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:18.821424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.826887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fad078-a735-470a-9ec4-e04d0e1e5182 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.856379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.864430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.865093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.865380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.865652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/2f33e1c573c044f086f99eb128c8f726/roles/c26057e0945745a39399a115b756ea2b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.884751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=2f33e1c573c044f086f99eb128c8f726, role_id=c26057e0945745a39399a115b756ea2b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.886310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:18.932362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a9dc-fb8a-43fb-8daa-a0e651c2beae None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:18.938606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 308/603] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:18 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/2f33e1c573c044f086f99eb128c8f726/roles/c26057e0945745a39399a115b756ea2b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:18.939605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:18.974735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:18.989687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:18.990309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:18.990676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:18.991022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:18.992314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:18.992640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:18.994717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.014773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 296/604] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:19.017083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.074636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6706546-7c7e-4c41-877e-1ed21339d3a9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.115894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.118147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.118446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.118774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.123246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:19.131717 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:19.135461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:19.139123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 309/605] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:19.139980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.148975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1bfa8-443d-42ed-82ac-fb3e2f38939f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.190747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.190747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.190997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.191320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.192119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:19.192486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:19.194023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.206527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 297/606] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:19.207569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.267346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:19.287687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b498d-99fc-4a41-8c15-06ca074ed29f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.316267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389eadd-e6d5-4bee-9bc5-5f73beee7647 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.339866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.341196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.341906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.342383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.351283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.351935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:19.352046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.352296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.352672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.353441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:19.353800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:19.354479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.355831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:19.368740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:19.369653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:19.371144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:19.371657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 298/607] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:19.371657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.373498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 310/608] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:19.374489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.387316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:19.507368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440156-2a97-4026-899a-2d1986fb75ec None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.511750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07f6aa-5139-4f12-81cc-0d654bbd11db None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.545524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.546146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.546428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.546727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.547809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.549018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.549526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.550011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/639c0cb750744031b47ba3e48170cce1/roles/c26057e0945745a39399a115b756ea2b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.558510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:19.560128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.566231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=639c0cb750744031b47ba3e48170cce1, role_id=c26057e0945745a39399a115b756ea2b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:19.567922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.569475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:19.576824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:19.583909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:19.584806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:19.584806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:19.584806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:19.584806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 311/609] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:19.584806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.673096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 299/610] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:19 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/639c0cb750744031b47ba3e48170cce1/roles/c26057e0945745a39399a115b756ea2b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:19.675354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.708098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91677b44-a3b4-4751-b633-21ce0e70ac68 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.787870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.788403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.788684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.788957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.791765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:19.792089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:19.794165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.809445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 312/611] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:19.812254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:19.815749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.888336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91bb3-ac4e-4858-96e7-f310fe285444 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.898584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:19.935840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff4001-e355-4336-9a65-842b6bb36cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.936501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4001-e355-4336-9a65-842b6bb36cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.936831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4001-e355-4336-9a65-842b6bb36cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.938031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4001-e355-4336-9a65-842b6bb36cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.939144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4001-e355-4336-9a65-842b6bb36cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:19.939458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4001-e355-4336-9a65-842b6bb36cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:19.941082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4001-e355-4336-9a65-842b6bb36cd5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:19.942187 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28749-5ed6-4d14-83e6-a104ba300c29 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:19.955907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 300/612] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:19.957527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522b52-2420-4b14-9434-14fa49be569a None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.957873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522b52-2420-4b14-9434-14fa49be569a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.958151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522b52-2420-4b14-9434-14fa49be569a None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:19.960619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 301/613] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:19.961582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd522b52-2420-4b14-9434-14fa49be569a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:19.985520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466922ba-5cc0-4a15-8389-8e8072b5bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:19.986459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466922ba-5cc0-4a15-8389-8e8072b5bab7 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:19.986932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466922ba-5cc0-4a15-8389-8e8072b5bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:19.987354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466922ba-5cc0-4a15-8389-8e8072b5bab7 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/422f81200e5645668e123229a6f356d3/roles/c26057e0945745a39399a115b756ea2b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:20.002889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466922ba-5cc0-4a15-8389-8e8072b5bab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=422f81200e5645668e123229a6f356d3, role_id=c26057e0945745a39399a115b756ea2b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:20.004645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466922ba-5cc0-4a15-8389-8e8072b5bab7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:20.079258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd522b52-2420-4b14-9434-14fa49be569a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:20.085066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 313/614] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:19 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/422f81200e5645668e123229a6f356d3/roles/c26057e0945745a39399a115b756ea2b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:20.087450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79134d6c-4caa-4195-9df2-df8ebe9ba4d4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:20.088016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79134d6c-4caa-4195-9df2-df8ebe9ba4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:20.088501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79134d6c-4caa-4195-9df2-df8ebe9ba4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:20.099168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:20.117469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-79134d6c-4caa-4195-9df2-df8ebe9ba4d4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:20.118857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02751969-1796-454f-8844-153abf492011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:20.119536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-79134d6c-4caa-4195-9df2-df8ebe9ba4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:20.119650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02751969-1796-454f-8844-153abf492011 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:20.119871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02751969-1796-454f-8844-153abf492011 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:20.120147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02751969-1796-454f-8844-153abf492011 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:20.129247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02751969-1796-454f-8844-153abf492011 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:20.130706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02751969-1796-454f-8844-153abf492011 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-02751969-1796-454f-8844-153abf492011 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:20.144387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:20.145611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:20.146827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:20.149369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 302/615] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:12:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:20.155668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02751969-1796-454f-8844-153abf492011 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:20.216496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 314/616] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:20.284169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02751969-1796-454f-8844-153abf492011 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:20.321193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:20.322064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:20.322492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:20.322938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:20.324389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:20.324867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:20.326464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:20.345802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 303/617] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:12:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:20.351482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79134d6c-4caa-4195-9df2-df8ebe9ba4d4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:20.399083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:20.464371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79134d6c-4caa-4195-9df2-df8ebe9ba4d4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:20.514245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:20.515132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:20.515597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:20.516016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:20.547789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:20.589202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:20.590985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:20.615168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:20.635033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:20.698199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 315/618] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:12:20.703958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:20.749457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3c1e9-44db-4546-81cf-088f37f6cd26 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:20.786424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:20.787924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:20.788462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:20.790064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:20.802376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:20.816305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:20.817523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:20.818732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 304/619] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:20.823476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:20.865834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5cb9a-cbd2-4ba1-8506-c91ec1fdc431 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:20.929221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:20.929851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:20.930130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:20.930401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:20.943962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:20.947222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:20.950170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:20.976214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:20.977474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:20.978509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:20.978509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:20.978509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:20.978509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 316/620] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:20.986697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:20.999326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:21.013966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a40873-d11b-4f75-aecf-98731b40a05d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.074884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.075671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.076062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.076441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.077212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:21.077623 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.079304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.100846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 305/621] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:21.107000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.126097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27015036-79af-4573-b34c-bf0dcd9c056d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.173450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf27530-a16e-4fa2-a2ba-a17ea0368b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.174076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf27530-a16e-4fa2-a2ba-a17ea0368b30 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.174799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf27530-a16e-4fa2-a2ba-a17ea0368b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.175073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf27530-a16e-4fa2-a2ba-a17ea0368b30 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.176002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf27530-a16e-4fa2-a2ba-a17ea0368b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:21.176304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf27530-a16e-4fa2-a2ba-a17ea0368b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.178312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf27530-a16e-4fa2-a2ba-a17ea0368b30 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.199844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 317/622] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:21.201169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6db39d-3879-4a79-aaed-bdb3f117846d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.201455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6db39d-3879-4a79-aaed-bdb3f117846d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.201730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6db39d-3879-4a79-aaed-bdb3f117846d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.203362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 318/623] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:21.204074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6db39d-3879-4a79-aaed-bdb3f117846d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.237590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50c41-6132-48db-9b92-247af85c6bf9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.271163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:21.277235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca7e01-4b97-4f9b-9758-d4bdfc3c1a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.277671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca7e01-4b97-4f9b-9758-d4bdfc3c1a90 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.277958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca7e01-4b97-4f9b-9758-d4bdfc3c1a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.278229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca7e01-4b97-4f9b-9758-d4bdfc3c1a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.290042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca7e01-4b97-4f9b-9758-d4bdfc3c1a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.292943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca7e01-4b97-4f9b-9758-d4bdfc3c1a90 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-4bca7e01-4b97-4f9b-9758-d4bdfc3c1a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:21.307575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:21.308695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:21.309607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:21.309607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:21.309607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:21.309607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 306/624] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:21.310550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319eaac-3ff6-4919-a438-fb63ed331630 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.310692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319eaac-3ff6-4919-a438-fb63ed331630 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.311038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319eaac-3ff6-4919-a438-fb63ed331630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.331065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-8319eaac-3ff6-4919-a438-fb63ed331630 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:21.332318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8319eaac-3ff6-4919-a438-fb63ed331630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:21.332966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6db39d-3879-4a79-aaed-bdb3f117846d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.334549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:21.378082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.379287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.379707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.380570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.390016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.391927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:21.405170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:21.406114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:21.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 319/625] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:21.407524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.414068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 307/626] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:21.415002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319eaac-3ff6-4919-a438-fb63ed331630 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.545292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319eaac-3ff6-4919-a438-fb63ed331630 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.614837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.615532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.616098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.616098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.616810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:21.617120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.617829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19adee-5851-4181-9160-f1d34fd02ef4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.618559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.625801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:21.632731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 308/627] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:21.632958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:21.646169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.715334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.715934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.716211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.716478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.717112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:21.717407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.718788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.749095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 320/628] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:21.759587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.764369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa39ca4-1c4f-46c0-8a43-f4abd4be8f00 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.807041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.808583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.809066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.809538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.810474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=0ed16fea638a45938eee8cdf234333f1) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:21.811016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.812683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.819150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 309/629] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/role_assignments?user.id=0ed16fea638a45938eee8cdf234333f1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:21.820209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.890094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:21.909279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a583-fab3-4bd0-aa05-536bafc9c3a2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.949873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a4ae-f381-4b3c-acda-cf6bc0c3de8d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:21.950910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65fffe-787e-4b69-82b9-e1536f2ba760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.951678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65fffe-787e-4b69-82b9-e1536f2ba760 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.952033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65fffe-787e-4b69-82b9-e1536f2ba760 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.952539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65fffe-787e-4b69-82b9-e1536f2ba760 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.960882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65fffe-787e-4b69-82b9-e1536f2ba760 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:21.962514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65fffe-787e-4b69-82b9-e1536f2ba760 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-cf65fffe-787e-4b69-82b9-e1536f2ba760 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:21.975832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:21.976884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:21.977810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 321/630] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:21.980158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b6ed-c96c-4785-84d5-dbd54002bd6c None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:21.980158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b6ed-c96c-4785-84d5-dbd54002bd6c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:21.980158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862b6ed-c96c-4785-84d5-dbd54002bd6c None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:21.986581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 322/631] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:21.988132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b6ed-c96c-4785-84d5-dbd54002bd6c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:21.998835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145012fd-1477-4a08-828c-1cfbbda5936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:21.999739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145012fd-1477-4a08-828c-1cfbbda5936e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.000200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145012fd-1477-4a08-828c-1cfbbda5936e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.000648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145012fd-1477-4a08-828c-1cfbbda5936e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.015573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145012fd-1477-4a08-828c-1cfbbda5936e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-145012fd-1477-4a08-828c-1cfbbda5936e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:22.022192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:22.023163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:22.024001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 310/632] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:22.025266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0fb42-f8fa-4817-8d3b-c9bf01674992 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.025266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0fb42-f8fa-4817-8d3b-c9bf01674992 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.025441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0fb42-f8fa-4817-8d3b-c9bf01674992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.034465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:22.047043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-84a0fb42-f8fa-4817-8d3b-c9bf01674992 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:22.049081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-84a0fb42-f8fa-4817-8d3b-c9bf01674992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:22.129574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862b6ed-c96c-4785-84d5-dbd54002bd6c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.134280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 311/633] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:22.135479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0fb42-f8fa-4817-8d3b-c9bf01674992 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.172900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.173958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.174594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.174988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.175827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:22.176234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.177819 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.198894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 323/634] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 12:12:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:22.201476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:22.215836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.254397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0fb42-f8fa-4817-8d3b-c9bf01674992 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.299934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.301250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.301676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.302356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.303194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:22.303882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.306129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.314623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:22.318845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 312/635] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:22.324015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.325618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910f70b-6840-400d-b6e4-4647147ed6d3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.386864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.387733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.388121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.388502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.408568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.410020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.445879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153002b-c0ab-44ff-9f7e-980fa0ce5ac8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:22.447708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:22.448651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:22.449543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:22.449543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:22.449543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:22.449543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 324/636] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:22.451067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.467517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:22.497106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.498007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.498431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.499408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.516490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.518706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:22.536780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:22.537943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:22.538904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:22.538904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:22.538904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:22.538904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 313/637] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:22.539135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.568176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720bb82-7a83-4014-9647-30e0c4cc2446 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.605970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.606591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.606906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.607485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.617519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:22.622682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:22.623619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:22.624547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 325/638] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:22.625272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.628101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:22.652603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16aeb8d-7252-42d6-9fcc-c596f886c92f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.689929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c053cc-9c9e-4d3d-bf17-bc06dc24a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.690451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c053cc-9c9e-4d3d-bf17-bc06dc24a407 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.690754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c053cc-9c9e-4d3d-bf17-bc06dc24a407 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.691028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c053cc-9c9e-4d3d-bf17-bc06dc24a407 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.691654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c053cc-9c9e-4d3d-bf17-bc06dc24a407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:22.691950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c053cc-9c9e-4d3d-bf17-bc06dc24a407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.693303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c053cc-9c9e-4d3d-bf17-bc06dc24a407 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.709121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 314/639] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:22.709121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de319-0939-4f39-82d8-bf9607c67c69 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.709121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de319-0939-4f39-82d8-bf9607c67c69 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.709121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de319-0939-4f39-82d8-bf9607c67c69 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.709121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 315/640] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:22.709121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968de319-0939-4f39-82d8-bf9607c67c69 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.735194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07938a-493a-4d6e-aafb-599912dadb50 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.736827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:22.772851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.773440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.773713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.773978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.774612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:22.774959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.776346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.788839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 326/641] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:22.789637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.822997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968de319-0939-4f39-82d8-bf9607c67c69 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.855413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:22.868625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a326c-13b2-4ca3-8bf9-c2581a3da54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.869648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a326c-13b2-4ca3-8bf9-c2581a3da54c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.869981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a326c-13b2-4ca3-8bf9-c2581a3da54c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.870280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a326c-13b2-4ca3-8bf9-c2581a3da54c None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.870966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a326c-13b2-4ca3-8bf9-c2581a3da54c None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:22.871264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a326c-13b2-4ca3-8bf9-c2581a3da54c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.872976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a326c-13b2-4ca3-8bf9-c2581a3da54c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.881697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 316/642] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:22.883021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657e244-ec10-4095-80a2-2c9080e82406 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.883410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657e244-ec10-4095-80a2-2c9080e82406 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.883774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657e244-ec10-4095-80a2-2c9080e82406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.899803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a29dd-f526-435a-b355-ce4803343f76 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:22.901915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-6657e244-ec10-4095-80a2-2c9080e82406 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:22.903709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6657e244-ec10-4095-80a2-2c9080e82406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:22.935606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:22.936339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:22.936689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:22.937024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:22.943811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:22.945273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:22.956724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:22.957856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:22.959034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:22.959927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 327/643] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:22.960653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:22.982690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 317/644] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:22.982690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657e244-ec10-4095-80a2-2c9080e82406 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.059005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:23.084578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bf5b-69c4-4b9f-88e6-092f92924782 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.088771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657e244-ec10-4095-80a2-2c9080e82406 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.113828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:23.127095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.127720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.127986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.128249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.139762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.141192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.148410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.149043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.149316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.149584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:23.157696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:23.158642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:12:23.159796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:23.161136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.162546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 328/645] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:23.162675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.163976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:23.180598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:23.181556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:23.183120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:23.183120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:23.183120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:23.183120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 318/646] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:23.183120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.271286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:23.273555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4917b8f-f9de-4984-88a6-96ecc69ad679 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.305050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36fac-73e4-453f-854f-27c835f0ce08 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.308616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.309209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.309484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.309751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.310428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:23.310757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.312171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.324563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 329/647] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:23.325582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.345023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.345679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.346064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.346336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.347219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:23.347521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.349032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.367526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 319/648] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:23.368316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.373995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:23.439616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7fbce-2329-4a14-8911-54c634844b51 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.475470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.476074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.476353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.476647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.477283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:23.477582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.479005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.481346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa5568-a8b1-4014-9190-55e9b766bf05 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.490914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 330/649] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:23.491696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.515672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:23.526322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.526322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.526322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.526322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.537286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:23.544769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:23.545892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:23.546977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:23.554225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 320/650] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:23.555122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.595850 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc663711-7a4b-4ef2-9cdd-fb26fbb45417 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.597571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:23.632664 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.633669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.633943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.634209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/909708a2a44a48e7b62a6f6a8d792f11/roles/c26057e0945745a39399a115b756ea2b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.648133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=909708a2a44a48e7b62a6f6a8d792f11, role_id=c26057e0945745a39399a115b756ea2b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.649545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.659550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77966416-873c-48ee-94a7-6d432e66bc07 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.693396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.693991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.694266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.694534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.721375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 331/651] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:23 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/909708a2a44a48e7b62a6f6a8d792f11/roles/c26057e0945745a39399a115b756ea2b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:23.722197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.775088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:23.783754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.785226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.864926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705a49-7549-4395-878b-e84a7b7ad0c8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:23.900247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 321/652] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:23.901948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.916263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:23.917356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:23.917841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:23.918167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:23.926523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:23.928085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:23.941472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:23.942633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:12:23.943894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:23.945096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 332/653] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:23.945999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:23.965999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.006156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.019137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6dd3f-c8cc-468f-8cf7-97622dbb59ec None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.065052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308b574-b22e-43c4-a9b0-2ea410812357 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.066269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.067009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.067490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.067828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.069375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:24.069747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.071215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.084111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 322/654] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:24.087642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.123344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24f0ef-ede9-46ee-8375-67e56ef9eef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.123977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24f0ef-ede9-46ee-8375-67e56ef9eef3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.124250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24f0ef-ede9-46ee-8375-67e56ef9eef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.124516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24f0ef-ede9-46ee-8375-67e56ef9eef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.140112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24f0ef-ede9-46ee-8375-67e56ef9eef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.142223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24f0ef-ede9-46ee-8375-67e56ef9eef3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fc24f0ef-ede9-46ee-8375-67e56ef9eef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:24.160618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:24.161746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:24.162816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:24.162816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:24.162816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:24.162816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 333/655] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:24.163107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015064a6-beb3-4e43-996d-d98805b40d8b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.163348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015064a6-beb3-4e43-996d-d98805b40d8b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.163624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015064a6-beb3-4e43-996d-d98805b40d8b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.164831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 334/656] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:24.165827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015064a6-beb3-4e43-996d-d98805b40d8b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.224959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6db60-0390-41c5-b643-c192e5619d10 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.247380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.274416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.275231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.275608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.275954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.276763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:24.277134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.278645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.303436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015064a6-beb3-4e43-996d-d98805b40d8b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.304378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 323/657] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:24.305291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.316469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.346816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123e2eb-8cdd-4c63-ac0d-04e0daa2c3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.347816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123e2eb-8cdd-4c63-ac0d-04e0daa2c3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.348329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123e2eb-8cdd-4c63-ac0d-04e0daa2c3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.348794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123e2eb-8cdd-4c63-ac0d-04e0daa2c3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.363782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123e2eb-8cdd-4c63-ac0d-04e0daa2c3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.365263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123e2eb-8cdd-4c63-ac0d-04e0daa2c3a2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d123e2eb-8cdd-4c63-ac0d-04e0daa2c3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:24.381194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:24.382309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:24.383376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:24.383376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:24.383376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:24.383376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 335/658] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:24.383913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bbe92-9add-4b6a-85cf-33fc39443756 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.384201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bbe92-9add-4b6a-85cf-33fc39443756 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.384473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bbe92-9add-4b6a-85cf-33fc39443756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.403108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-115bbe92-9add-4b6a-85cf-33fc39443756 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:24.404482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-115bbe92-9add-4b6a-85cf-33fc39443756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:24.444938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a7deb-9997-42ba-bfea-3e16382175c7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.491045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 336/659] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:24.492037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bbe92-9add-4b6a-85cf-33fc39443756 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.513898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.514861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.515406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.515896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.516779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:24.517507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.523254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.539753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 324/660] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:24.541101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.541567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.597728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bbe92-9add-4b6a-85cf-33fc39443756 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.619022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.634443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.635360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.635642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.635940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.649360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c6302-b1a9-4114-b13f-e2a7df7b5265 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.686538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.687959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.688438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.688968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.689896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:24.690388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.692037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.704982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 325/661] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:24.706015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.714352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.716077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.768109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.822017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2c7ef-456e-40cd-9904-d377a7a0a4f2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.826024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 337/662] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:24.831332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.850484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:24.859153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.874856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.874856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.874856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.874856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.874856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:24.879634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:24.880505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 326/663] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:24.881437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:24.943826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19306a-a0fa-4a5d-83f6-2c0fb3f4a2bc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.978453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:24.979102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:24.979400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:24.980204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:24.988012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:24.988225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf0d3e-e924-4711-86a4-568575688bbb None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:24.989427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:25.000935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:25.001981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 338/664] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:25.003345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.011532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.027580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.028251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.028525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.029043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.038979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:25.044587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:25.045589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:25.046444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 327/665] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:25.047678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.100580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.111756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912cc56-0638-4cb6-923e-6b550360f382 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.146023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.146644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.146948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.147221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.153444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2826-092f-44f6-aadd-0a78916e26e0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.162839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.164383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:25.180078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:25.181225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:25.182333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:25.182333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:25.182333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:25.186156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 339/666] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:25.187101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.200494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.201167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.201447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.201716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.202522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:25.202834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.204300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.221447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 328/667] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:25.221447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.253872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.298237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d2028-7e60-4d95-b0d8-26bb37f67943 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.328364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe02e5c-362c-4afc-8bdc-85681d5b9459 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.333478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.334081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.334360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.334676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.335425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:25.335727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.337141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.338229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.355054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 340/668] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:25.355809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.370307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.371211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.371789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.372244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.373160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:25.373640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.375484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.385112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 329/669] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:25.386796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.472497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cffb66-27eb-402a-aa2f-0b01fef10306 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.489732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.504307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8061d28-8146-4912-a7d7-f1b712182fc3 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.507494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.508354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.508456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.508840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.509488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:25.509789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.511696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.523123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 341/670] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:25.523731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.544700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.545294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.545575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.545846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.546563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=2f33e1c573c044f086f99eb128c8f726) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:25.546904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.548331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.554434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 330/671] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/role_assignments?user.id=2f33e1c573c044f086f99eb128c8f726&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:25.555263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.596252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.675453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af67584-c2b9-4d2c-ba90-d0abb34bacab None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.676631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c6f8-e492-447f-930e-5e0c9d19472f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.721764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.722411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.722721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.722986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.726317 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70daf3c-6bed-4b06-ae61-e479bae7d6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.727007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daf3c-6bed-4b06-ae61-e479bae7d6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.727291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daf3c-6bed-4b06-ae61-e479bae7d6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.727565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daf3c-6bed-4b06-ae61-e479bae7d6e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.736738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.738614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:25.756732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:25.757719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:25.759107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:25.759107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:25.759107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:25.759107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 331/672] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:25.759107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.785497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.809856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70daf3c-6bed-4b06-ae61-e479bae7d6e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.811368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70daf3c-6bed-4b06-ae61-e479bae7d6e2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.852830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:25.867698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb95a5-93d7-43e3-90bb-6ea2e2542c8d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:25.904134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:25.904768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.905048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.905318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.913236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:25.914724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:25.925078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 342/673] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:25.936457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25f41-76a8-400f-bd58-3e9e76101b66 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:25.936457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25f41-76a8-400f-bd58-3e9e76101b66 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:25.936457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25f41-76a8-400f-bd58-3e9e76101b66 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:25.936457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 343/674] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:25.936457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c25f41-76a8-400f-bd58-3e9e76101b66 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:25.936873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:25.937692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:25.938581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 332/675] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:25.939649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.021442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.045756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c25f41-76a8-400f-bd58-3e9e76101b66 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.046775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71803cf4-ac36-45eb-afa3-8dd5ca0b1156 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.081394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1be6ce-53de-4130-912e-f3b8b7af42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.081986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1be6ce-53de-4130-912e-f3b8b7af42c0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.082260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1be6ce-53de-4130-912e-f3b8b7af42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.082525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1be6ce-53de-4130-912e-f3b8b7af42c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.083191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1be6ce-53de-4130-912e-f3b8b7af42c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:26.083392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.083503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1be6ce-53de-4130-912e-f3b8b7af42c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.084006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.084289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.084561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.084834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1be6ce-53de-4130-912e-f3b8b7af42c0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.096023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.101540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 344/676] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:26.101668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.101765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ecd9-0768-438e-9eb8-3f1f68198c85 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.101765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ecd9-0768-438e-9eb8-3f1f68198c85 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.101765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ecd9-0768-438e-9eb8-3f1f68198c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.109191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:26.117340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:26.124951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:26.126043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:26.126043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:26.126043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:26.126233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8d78ecd9-0768-438e-9eb8-3f1f68198c85 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:26.126314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 333/677] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:26.126314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.126444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8d78ecd9-0768-438e-9eb8-3f1f68198c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:26.195217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 345/678] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:26.195993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ecd9-0768-438e-9eb8-3f1f68198c85 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.248200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad930da-fbf3-4bf3-82e6-8282d82bf3a9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.270226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.309020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.310040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.310261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ecd9-0768-438e-9eb8-3f1f68198c85 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.310924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.311362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.312973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:26.313454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.315931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.337412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 334/679] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:26.337412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.358391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.359374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.359884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.360372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.367937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.369618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:26.381879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:26.383042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:26.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:26.391347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 346/680] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:26.392403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.398415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.441311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b178299-6c1f-449d-b3e7-b0b8f2783130 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.477703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.478230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.478507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.478806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.479543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:26.479991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.481309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.498169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 335/681] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:26.498169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.505806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dba20-e2c3-4735-985e-74bd8cb72fa4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.533532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.543760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.549725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.549725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.549725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.549725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=639c0cb750744031b47ba3e48170cce1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:26.549725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.549725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.562677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 347/682] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/role_assignments?user.id=639c0cb750744031b47ba3e48170cce1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:26.562677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.629227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc749986-ae41-4038-8333-89a8f21d718d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.655047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.674871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.678833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.678833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.678833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.678833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:26.678833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.678833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.688599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9eff-068b-4285-8fda-3cf689353b62 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.700940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 336/683] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:26.701152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.741280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.741920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.742199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.742467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.755754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:26.760901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:26.761872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:26.762824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 348/684] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:26.764068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.826891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c26f6-3f20-4ecf-98ab-d71a0fe295d7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.863760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.864301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.864424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.864650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.864920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:26.873306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:26.874759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:26.886558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:26.887528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:26.888450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 337/685] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:26.888950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:26.896118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea72622-90b7-4ef8-9f86-22d6a512529f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:26.931819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:26.943502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:26.944376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:26.944976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:26.945455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.007521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900222cc-0be7-4450-8ce0-c3a98d146e02 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.039748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.041522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.043532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.044303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.044673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.045031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.046316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:27.046316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.047556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.061528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 338/686] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:27.062499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.098749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:27.164646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 349/687] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:27.165607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.191134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:27.196470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9c850-1b96-4429-bce4-ed47315f96d8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.237833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.238547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.238928 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.239241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.240000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:27.240364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.242031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.261211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 339/688] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:27.262082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.287063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ec0a6-4494-4c96-949b-4a91b9c12e09 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.324411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.325414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.325865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.326386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.337233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.339008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:27.354449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:27.355462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:27.356338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:27.356338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:27.356338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:27.358992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 350/689] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:27.360093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.365829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:27.378784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67770cd-b2c8-4ca4-b0b8-13d27f25b535 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.416707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.418324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.418895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.419454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.431772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.434039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:27.450306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:27.451298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:27.452179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:27.452179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:27.452179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:27.454426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 340/690] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:27.455547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.470844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:27.476529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309963d-5ecf-4601-8235-aefa7aff7cd6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.512455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.512975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.513249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.513516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.514238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=422f81200e5645668e123229a6f356d3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:27.514541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.515992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.526726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 351/691] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/role_assignments?user.id=422f81200e5645668e123229a6f356d3&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:27.526726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.588293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03728d15-06d0-4124-8933-a824f5484f76 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.598336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:27.629888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e25bb-b0e9-4fad-b8a0-683124b49791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.630697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e25bb-b0e9-4fad-b8a0-683124b49791 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.631088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e25bb-b0e9-4fad-b8a0-683124b49791 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.631468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e25bb-b0e9-4fad-b8a0-683124b49791 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.632219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e25bb-b0e9-4fad-b8a0-683124b49791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:27.632620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e25bb-b0e9-4fad-b8a0-683124b49791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.634314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e25bb-b0e9-4fad-b8a0-683124b49791 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.640383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b14d4-7648-427a-834b-4715da26b31b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.650081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 341/692] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:27.656113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f09e-93d5-42f6-ae4a-5e70a49cdb04 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.656537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f09e-93d5-42f6-ae4a-5e70a49cdb04 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.656909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14f09e-93d5-42f6-ae4a-5e70a49cdb04 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.658924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 342/693] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:27.660874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f09e-93d5-42f6-ae4a-5e70a49cdb04 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.681102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09466c6-2314-4f4b-b248-e41231556708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.682005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09466c6-2314-4f4b-b248-e41231556708 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.682453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09466c6-2314-4f4b-b248-e41231556708 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.682933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09466c6-2314-4f4b-b248-e41231556708 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.683817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09466c6-2314-4f4b-b248-e41231556708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:27.684303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09466c6-2314-4f4b-b248-e41231556708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.685936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09466c6-2314-4f4b-b248-e41231556708 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.700971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 352/694] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:27.702609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdf36-c158-4ab7-b9a5-a47aaf4d2f52 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.703132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdf36-c158-4ab7-b9a5-a47aaf4d2f52 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.703545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdf36-c158-4ab7-b9a5-a47aaf4d2f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.723523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-627fdf36-c158-4ab7-b9a5-a47aaf4d2f52 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:27.724951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-627fdf36-c158-4ab7-b9a5-a47aaf4d2f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:27.741946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:27.785895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14f09e-93d5-42f6-ae4a-5e70a49cdb04 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.812094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 353/695] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:27.813222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fdf36-c158-4ab7-b9a5-a47aaf4d2f52 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.839933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.840564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.840899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.841177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.859343 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.860857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.876243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:27.879543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:27.881019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:27.882159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 343/696] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:27.888087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:27.933579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fdf36-c158-4ab7-b9a5-a47aaf4d2f52 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:27.969934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:27.970562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:27.970867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:27.971136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:27.978228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:27.980387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:27.991949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:27.992860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:27.993943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 354/697] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:27.999872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.116548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db5ea-fad0-45d6-9c01-6cc59cdc061a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:28.174689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15345066-01bd-4708-8996-8cc513ce3a7f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:28.177442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:28.178921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.179203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.179473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.180351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:28.180659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:28.182197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:28.187070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:28.195244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:28.206287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 344/698] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:28.211660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.214929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:28.221400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.221400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.221400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.221400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:28.221400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:28.221400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:28.232450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 355/699] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:28.233680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.336424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff683d9b-6005-4839-b3c7-cf6a06761efb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:28.347690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48edef-a53c-438a-92ca-5bfd9921c545 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:28.373240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:28.374127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.374617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.375156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.391921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:28.392547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.392825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.393352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.404969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:28.406760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:28.430992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:28.434250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:28.435211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:28.436101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:28.436101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:28.436101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:28.442575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 345/700] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:28.443706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.457148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:28.459395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:28.489859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:28.549726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbe8d3-c29e-420c-83dc-83234cfd1825 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:28.585049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b727-5724-4cf7-820d-36fb0a8497fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:28.586008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b727-5724-4cf7-820d-36fb0a8497fe None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.586494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b727-5724-4cf7-820d-36fb0a8497fe None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.586987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b727-5724-4cf7-820d-36fb0a8497fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.587930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b727-5724-4cf7-820d-36fb0a8497fe None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=0ed16fea638a45938eee8cdf234333f1) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:28.588497 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b727-5724-4cf7-820d-36fb0a8497fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:28.590734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b727-5724-4cf7-820d-36fb0a8497fe None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:28.596718 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 356/701] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:12:28.597501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.600834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 346/702] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:28 2022] GET /identity/v3/role_assignments?user.id=0ed16fea638a45938eee8cdf234333f1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:28.602501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00062f8-1a56-4aff-84f3-6078de9325e9 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.602815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00062f8-1a56-4aff-84f3-6078de9325e9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.603135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00062f8-1a56-4aff-84f3-6078de9325e9 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.605036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 347/703] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:28.605679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00062f8-1a56-4aff-84f3-6078de9325e9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.712330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:28.738177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00062f8-1a56-4aff-84f3-6078de9325e9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:28.743053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:28.750939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79114ef8-98c4-4b45-975d-a4df027c52c4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:28.786214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:28.787051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.787396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.787806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.789824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24ecab-cccc-489f-bf5b-812b48653254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:28.790532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ecab-cccc-489f-bf5b-812b48653254 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.790922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ecab-cccc-489f-bf5b-812b48653254 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.791270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ecab-cccc-489f-bf5b-812b48653254 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.792514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24ecab-cccc-489f-bf5b-812b48653254 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:28.792819 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24ecab-cccc-489f-bf5b-812b48653254 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:28.794239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24ecab-cccc-489f-bf5b-812b48653254 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:28.802568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:28.805260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:28.811367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 357/704] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:28.811367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8257a5-0354-4b42-8eb5-1cfbbc0b776e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:28.811367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8257a5-0354-4b42-8eb5-1cfbbc0b776e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:28.811367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8257a5-0354-4b42-8eb5-1cfbbc0b776e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:28.827914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4b8257a5-0354-4b42-8eb5-1cfbbc0b776e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:28.828796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:28.829739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:28.830642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:28.830642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:28.830642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:28.830843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4b8257a5-0354-4b42-8eb5-1cfbbc0b776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:28.834062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 348/705] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:28.835210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.910737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 358/706] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:28.912238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8257a5-0354-4b42-8eb5-1cfbbc0b776e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:28.983409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:29.018809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8257a5-0354-4b42-8eb5-1cfbbc0b776e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.030344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805715ae-af4b-437b-994e-b31571ea5b25 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.055977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.056580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.056856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.057127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.057905 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:29.058349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.059789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.072384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 359/707] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:29.082835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:29.085958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.086638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.086946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.087219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.087860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.089860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.183577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 349/708] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:12:28 2022] POST /identity/v3/users => generated 254 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:29.205903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288749e-a718-49ab-a19d-e2596d2b5bd9 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.242102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:29.253677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.254318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.254643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.254944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.258909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:29.271987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.273474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.282230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:29.297787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:29.300806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:29.301782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 360/709] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:29.305670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:29.420852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301d3be-2309-45aa-97a3-489c9d5cd87b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.450632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70396690-13c4-4a71-b687-cae788fb3db9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.466403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.467386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.467721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.468060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.468822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:29.469190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.470863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.485731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 361/710] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:29.492458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:29.500164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.500864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.501141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.501411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] PATH_INFO: `/v3/services/network` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.502566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.504431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:29.517692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:29.518746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:29.519771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 350/711] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:29 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:29.527052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:29.551610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:29.624543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef97c8-5fe4-4669-add3-2142b67d1e2d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.638926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:29.663905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a02ba-9336-42ab-b1f2-a3626c20b86f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.680534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.681649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.682068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.683086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.684115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=909708a2a44a48e7b62a6f6a8d792f11) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:29.685035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.686898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.696864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 362/712] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:29 2022] GET /identity/v3/role_assignments?user.id=909708a2a44a48e7b62a6f6a8d792f11&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:29.712283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83125d9a-2897-436c-8618-0df3fadee41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.713387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125d9a-2897-436c-8618-0df3fadee41f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.713853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125d9a-2897-436c-8618-0df3fadee41f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.714221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125d9a-2897-436c-8618-0df3fadee41f None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.715057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83125d9a-2897-436c-8618-0df3fadee41f None admin] RBAC: Adding query filter params (name=network) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:29.715565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83125d9a-2897-436c-8618-0df3fadee41f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.717071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83125d9a-2897-436c-8618-0df3fadee41f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.723013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 351/713] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:29.728650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:29.771859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e23da-b48f-40a3-a134-07ef266cc10e None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.772053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e23da-b48f-40a3-a134-07ef266cc10e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.772324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e23da-b48f-40a3-a134-07ef266cc10e None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.774633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 352/714] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:29.780348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f347f-bd9b-4b6c-8485-a9e49d9c00c6 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.780639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f347f-bd9b-4b6c-8485-a9e49d9c00c6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.780903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f347f-bd9b-4b6c-8485-a9e49d9c00c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.802527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c42f347f-bd9b-4b6c-8485-a9e49d9c00c6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:29.804364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c42f347f-bd9b-4b6c-8485-a9e49d9c00c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:29.910580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 353/715] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:12:29.938801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13b81-0d82-43b0-952c-de5dcccd4d7e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:29.952978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:29.977458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:29.978250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:29.978847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:29.979729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:29.980433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] RBAC: Adding query filter params (type=network) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:29.980810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:29.982492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:29.988633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 363/716] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:30.224135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f347f-bd9b-4b6c-8485-a9e49d9c00c6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:30.299837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:30.342784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f347f-bd9b-4b6c-8485-a9e49d9c00c6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:30.386209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:30.386858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:30.387132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:30.387400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:30.387980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:30.389334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:30.403994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:30.405013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:30.405848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:30.406584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 354/717] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 24 12:12:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:30.410959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:30.569403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c998d3-2ebd-48c5-8e6c-67bfdf8bce58 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:30.615499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cf48a-8deb-4b1d-b983-4ad4d2d36ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:30.616455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cf48a-8deb-4b1d-b983-4ad4d2d36ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:30.616962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cf48a-8deb-4b1d-b983-4ad4d2d36ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:30.617437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cf48a-8deb-4b1d-b983-4ad4d2d36ee0 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:30.618277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cf48a-8deb-4b1d-b983-4ad4d2d36ee0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:30.618805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cf48a-8deb-4b1d-b983-4ad4d2d36ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:30.620419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cf48a-8deb-4b1d-b983-4ad4d2d36ee0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:30.625477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 364/718] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 12:12:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:30.630711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:30.792295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7d8bf-bb38-4417-b7bd-a5e53dc68262 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:30.838041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8392c33-54f7-43dc-bb5f-b6de2199c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:30.838600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8392c33-54f7-43dc-bb5f-b6de2199c234 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:30.838906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8392c33-54f7-43dc-bb5f-b6de2199c234 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:30.839177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8392c33-54f7-43dc-bb5f-b6de2199c234 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:30.839809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8392c33-54f7-43dc-bb5f-b6de2199c234 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:30.840109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8392c33-54f7-43dc-bb5f-b6de2199c234 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:30.841475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8392c33-54f7-43dc-bb5f-b6de2199c234 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:30.842763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:30.848351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 355/719] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 12:12:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:30.856594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7129e-aa39-41e4-8013-0a69e6ce1ed8 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:30.857795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7129e-aa39-41e4-8013-0a69e6ce1ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:30.858236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7129e-aa39-41e4-8013-0a69e6ce1ed8 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:30.860109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 365/720] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:30.866222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635c570-756d-4f0a-bd48-ea03cf3ba835 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:30.866515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635c570-756d-4f0a-bd48-ea03cf3ba835 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:30.866833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635c570-756d-4f0a-bd48-ea03cf3ba835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:30.895753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0635c570-756d-4f0a-bd48-ea03cf3ba835 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:30.896995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0635c570-756d-4f0a-bd48-ea03cf3ba835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:31.022524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 356/721] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:31.375298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7129e-aa39-41e4-8013-0a69e6ce1ed8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:31.420180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:31.487867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7129e-aa39-41e4-8013-0a69e6ce1ed8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:31.530074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:31.530728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:31.531011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:31.531282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:31.542367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:31.547205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:31.548126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:31.549371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 366/722] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:31.555454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635c570-756d-4f0a-bd48-ea03cf3ba835 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:31.704983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635c570-756d-4f0a-bd48-ea03cf3ba835 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:31.739031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:31.749242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2daea2-b5b5-4862-afc0-93b37f153371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:31.749855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2daea2-b5b5-4862-afc0-93b37f153371 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:31.750136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2daea2-b5b5-4862-afc0-93b37f153371 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:31.750410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2daea2-b5b5-4862-afc0-93b37f153371 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:31.751110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2daea2-b5b5-4862-afc0-93b37f153371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:31.755465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2daea2-b5b5-4862-afc0-93b37f153371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:31.755465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2daea2-b5b5-4862-afc0-93b37f153371 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:31.764809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 357/723] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:31.770285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:31.897951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:31.902882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b47d6f-a3ec-4ae5-8d7e-e02a2849a453 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:31.952626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:31.953254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:31.953532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:31.953804 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:31.974198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:31.974198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:31.979612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f831e13-52b2-4c1f-a411-792236811e78 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:31.979958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f831e13-52b2-4c1f-a411-792236811e78 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:31.980242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f831e13-52b2-4c1f-a411-792236811e78 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:31.982256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 358/724] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:31.987313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971d6c7-db41-45be-9049-27858671c3c5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:31.987604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971d6c7-db41-45be-9049-27858671c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:31.987886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971d6c7-db41-45be-9049-27858671c3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:31.991778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:31.992799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:31.994039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:31.994039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:31.994039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:31.994039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 367/725] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:32.000892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.012021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3971d6c7-db41-45be-9049-27858671c3c5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:32.013495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3971d6c7-db41-45be-9049-27858671c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:32.087946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 359/726] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:12:32.145417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a53cb8-3a6b-423f-bdef-a8c77e459477 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:32.191341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c289b-78e7-4e83-b52f-8ecaaaadd306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:32.191945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c289b-78e7-4e83-b52f-8ecaaaadd306 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.192228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c289b-78e7-4e83-b52f-8ecaaaadd306 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.192502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c289b-78e7-4e83-b52f-8ecaaaadd306 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.193143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c289b-78e7-4e83-b52f-8ecaaaadd306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:32.193444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c289b-78e7-4e83-b52f-8ecaaaadd306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:32.194884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c289b-78e7-4e83-b52f-8ecaaaadd306 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:32.209966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 368/727] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:32.215386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971d6c7-db41-45be-9049-27858671c3c5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.326698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1834e97-9dac-4fc4-8895-d1438131afeb None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.326698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1834e97-9dac-4fc4-8895-d1438131afeb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.326698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1834e97-9dac-4fc4-8895-d1438131afeb None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.327046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 369/728] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:32.330937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24218d-a8b1-45da-a5bf-5d02cb0e68b9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.331224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24218d-a8b1-45da-a5bf-5d02cb0e68b9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.331502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24218d-a8b1-45da-a5bf-5d02cb0e68b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.347195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971d6c7-db41-45be-9049-27858671c3c5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:32.375051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:32.378528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9c24218d-a8b1-45da-a5bf-5d02cb0e68b9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:32.379770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9c24218d-a8b1-45da-a5bf-5d02cb0e68b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:32.387365 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:32.395710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:32.396630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.397071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.402948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.415459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:32.417493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:32.440845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:32.441833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 360/729] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:12:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:32.443219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.478703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 370/730] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:32.480081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492a1e6-9923-4947-aada-2150a6c093e6 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.480361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492a1e6-9923-4947-aada-2150a6c093e6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.480635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492a1e6-9923-4947-aada-2150a6c093e6 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.482221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 371/731] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:32.487109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492a1e6-9923-4947-aada-2150a6c093e6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.565169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d85d7-f8fd-4b0d-a99e-8a36e6c48bff None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:32.605616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492a1e6-9923-4947-aada-2150a6c093e6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:32.615557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579cd29-b09b-4512-8109-7440f5155eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:32.616896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579cd29-b09b-4512-8109-7440f5155eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.617766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579cd29-b09b-4512-8109-7440f5155eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.618229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579cd29-b09b-4512-8109-7440f5155eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.632697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579cd29-b09b-4512-8109-7440f5155eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-9579cd29-b09b-4512-8109-7440f5155eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:32.640587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:32.641518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:32.642382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:32.647432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:32.648281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.648692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 361/732] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:32.648987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.649454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.649909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24f633-faf7-4294-8c33-609ee417beb3 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.650188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24f633-faf7-4294-8c33-609ee417beb3 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.650455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24f633-faf7-4294-8c33-609ee417beb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.651605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:32.652116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:32.653743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:32.665579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:32.669494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1b24f633-faf7-4294-8c33-609ee417beb3 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:32.670681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1b24f633-faf7-4294-8c33-609ee417beb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:32.677512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 372/733] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:12:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:32.679152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.708006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:32.747802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 362/734] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:32.747802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24f633-faf7-4294-8c33-609ee417beb3 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.829435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27881ee-77cf-452e-a80c-eec13e241560 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:32.863404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24f633-faf7-4294-8c33-609ee417beb3 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:32.889422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:32.891195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.891982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.892460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.893714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:32.894560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:32.896758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:32.902604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:32.903456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:32.903800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:32.904166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:32.909083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 373/735] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:32.910993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.920537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:32.922140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:32.938083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:32.939124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:32.940043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:32.940043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:32.940043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:32.940043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 363/736] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:32.940980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:32.969513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:32.990123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:33.035487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ea67-1e75-404f-91b4-60bd09b713bb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.069055 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3a1c5-9327-4eab-aac6-d3f0950617d8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.071948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.071948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.072244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.072494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.073703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.075117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.104073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.104690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.104968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.105238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.116065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.117438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:33.131756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:33.137422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:33.138561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:33.138561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:33.138561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:33.138561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 364/737] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:33.138561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.174922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 374/738] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:12:32 2022] POST /identity/v3/services => generated 240 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:33.176453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.220146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:33.251526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec78879-4e31-4521-b3b2-007cd138dc88 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.299182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.299890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.300278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.300617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.301330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:33.301760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.304120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.324790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 365/739] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:33.325738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.332120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b8c72-1a1d-462b-9c3f-38f83bd2ae55 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.354003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:33.370596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f52dc-60d2-4f63-8843-329112ccd58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.371179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f52dc-60d2-4f63-8843-329112ccd58c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.371456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f52dc-60d2-4f63-8843-329112ccd58c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.371722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f52dc-60d2-4f63-8843-329112ccd58c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.384453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f52dc-60d2-4f63-8843-329112ccd58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a48f52dc-60d2-4f63-8843-329112ccd58c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:33.389963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:33.391142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:33.392202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:33.394981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 375/740] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:33.396573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb23408-aaa0-4406-a8a5-3bd9ea5df2a7 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.397252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb23408-aaa0-4406-a8a5-3bd9ea5df2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.397377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb23408-aaa0-4406-a8a5-3bd9ea5df2a7 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.402716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 376/741] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:33.402716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb23408-aaa0-4406-a8a5-3bd9ea5df2a7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.444045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b687b-2f22-4562-9085-400585c0d4fa None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.474571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:33.480827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.481433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.481713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.483421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.483421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:33.483421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.484759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.497195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 366/742] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:33.498120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.524062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb23408-aaa0-4406-a8a5-3bd9ea5df2a7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.569684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe72073-29f7-4f91-b37d-4b07e5c8dc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.571258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72073-29f7-4f91-b37d-4b07e5c8dc92 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.571543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72073-29f7-4f91-b37d-4b07e5c8dc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.571820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe72073-29f7-4f91-b37d-4b07e5c8dc92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.580783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe72073-29f7-4f91-b37d-4b07e5c8dc92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.582870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe72073-29f7-4f91-b37d-4b07e5c8dc92 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-bbe72073-29f7-4f91-b37d-4b07e5c8dc92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:33.597730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:33.598669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:33.599583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:33.600011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 377/743] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:33.601133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f16b-e47e-47b6-8fae-30b856fa3c92 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.601415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f16b-e47e-47b6-8fae-30b856fa3c92 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.601683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f16b-e47e-47b6-8fae-30b856fa3c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.608677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701bff-d60d-4e18-b6ab-0e2261370f5d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.619784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ce75f16b-e47e-47b6-8fae-30b856fa3c92 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:33.621015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ce75f16b-e47e-47b6-8fae-30b856fa3c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:33.645556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.646157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.646491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.646798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.647465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:33.647762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.649155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.657287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:33.664241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 367/744] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:33.664241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.698795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 378/745] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:33 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:12:33.699894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f16b-e47e-47b6-8fae-30b856fa3c92 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.719360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:33.779721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0101-5d67-4ce3-93c5-d447434fe348 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.819188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.819962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.820335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.820483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f16b-e47e-47b6-8fae-30b856fa3c92 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.820801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.829003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.830758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:33.842594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:33.843760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:33.845018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:33.850726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 368/746] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:33.850726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.861387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:33.862266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:33.862546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:33.862845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:33.863532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:33.864091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:33.866175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:33.884095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 379/747] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:33.884095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:33.920859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:33.984352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068cf1b-ec0b-41bd-b3d3-0fdc28cda33b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.996176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b1199-8244-465d-bef1-009c75d899c0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:33.998418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:34.020091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.020704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.020978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.021303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.031377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.042907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.043033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.043033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.043033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.043287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.044215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:34.044674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:34.046291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:34.047215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:34.048140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:34.048140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:34.048140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:34.048309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.052405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 369/748] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:34.053101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.070268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 380/749] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 12:12:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:34.071882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.157323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08234cd3-1404-43b4-bfbd-b9bfa028bef9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:34.174011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:34.187017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c816cd-95f0-4986-8019-3ddf3ee66bee None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:34.194541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.195274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.195558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.195829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/0ed16fea638a45938eee8cdf234333f1/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.197722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=0ed16fea638a45938eee8cdf234333f1, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.199227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.220250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.220842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.221132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.221380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.222020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.223412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.274590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 370/750] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:34 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/0ed16fea638a45938eee8cdf234333f1/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:34.283722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.313083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:34.323218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 381/751] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:12:34 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:34.323218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.487070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088031c-d061-4489-be27-d7460617d2dc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:34.544433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.544850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.545155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.545536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.546182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:34.546488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.547413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555ed9d-ec9e-484e-99ca-96ed517892db None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:34.547895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.564282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 371/752] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:34.569678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.591427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.592421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.592921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.593387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.609899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:34.656003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:34.668725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.670343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.677663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039c959-ea47-47dd-8a54-c2a2878cf157 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:34.714144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.715035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.715464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.715885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.724493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.726180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:34.739347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:34.740504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:34.741524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:34.743806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 372/753] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:34.750754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.782814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 382/754] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:12:34.788668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.835559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:34.857528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5a679-aead-486f-ad1f-e587bc833df3 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:34.894265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b86f8-d2da-4ec7-b6e7-a4131b6af84e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:34.895145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.896037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.896522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.897016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.897973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:34.898492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.900167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.910327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:34.920520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 373/755] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 12:12:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:34.926281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:34.933880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:34.933880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:34.946820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:34.946820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:34.946820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:34.947088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:34.968174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:34.969165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:34.970175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:34.970175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:34.970175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:34.970175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 383/756] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:34.970175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:35.091989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:35.098455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ae14d-8813-4a55-b7b1-aef6a4ae0e9c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:35.145234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:35.145838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.146117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.146386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.147051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:35.147350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:35.148729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:35.160835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926c5d2-ab5e-49d7-85d1-9fd9172ae777 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:35.165701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 384/757] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:35.167028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:35.247748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f03a4-c52e-4c3f-ab95-a2d5c6e3b2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:35.248366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f03a4-c52e-4c3f-ab95-a2d5c6e3b2cd None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.248657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f03a4-c52e-4c3f-ab95-a2d5c6e3b2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.248959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f03a4-c52e-4c3f-ab95-a2d5c6e3b2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.302220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15117b00-a4b8-4f0c-a764-8db0f0a10fdc None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:35.353490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b11fb3-0398-4ee4-a233-96a1140affd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:35.354153 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11fb3-0398-4ee4-a233-96a1140affd1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.354429 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11fb3-0398-4ee4-a233-96a1140affd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.354726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11fb3-0398-4ee4-a233-96a1140affd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.356511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:35.361067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f03a4-c52e-4c3f-ab95-a2d5c6e3b2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:35.363072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b11fb3-0398-4ee4-a233-96a1140affd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:35.363427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f03a4-c52e-4c3f-ab95-a2d5c6e3b2cd None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:35.364559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b11fb3-0398-4ee4-a233-96a1140affd1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a9b11fb3-0398-4ee4-a233-96a1140affd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:35.377040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:35.377960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:35.378914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:35.383137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 385/758] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:35.384178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68243703-d098-44c5-befe-3d098817cfae None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.384463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68243703-d098-44c5-befe-3d098817cfae None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.384796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68243703-d098-44c5-befe-3d098817cfae None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.390678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 386/759] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:35.390678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68243703-d098-44c5-befe-3d098817cfae None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:35.409729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:35.520979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 374/760] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:12:35.523412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0338b-273b-44c1-9653-72afaa7f8dcd None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.523976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0338b-273b-44c1-9653-72afaa7f8dcd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.524630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0338b-273b-44c1-9653-72afaa7f8dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.532494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68243703-d098-44c5-befe-3d098817cfae None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:35.547873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5fd0338b-273b-44c1-9653-72afaa7f8dcd None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:35.549627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5fd0338b-273b-44c1-9653-72afaa7f8dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:35.571618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:35.572526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.573552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.573918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.574833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:35.575366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:35.577221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:35.598731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 387/761] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:35.600143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:35.637872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 375/762] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:35.639262 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0338b-273b-44c1-9653-72afaa7f8dcd None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:35.670730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:35.748631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:35.755073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0338b-273b-44c1-9653-72afaa7f8dcd None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:35.809515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:35.810474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.811161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.811638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.812752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=44144b71637548c0b31368fc8114b6b6) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:35.813331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:35.815149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:35.826682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 376/763] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:35 2022] GET /identity/v3/role_assignments?user.id=44144b71637548c0b31368fc8114b6b6&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:35.845135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b30c-14b8-4356-be1a-5378e2b5724e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:35.906202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:35.906883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:35.907159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:35.907427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:35.927708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:35.929239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:35.947708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:35.949449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:35.950563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:35.950563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:35.950563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:35.950563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 388/764] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:35.954750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:35.993239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:36.114373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e03ff3-a4df-44f1-bd45-a3c93f5fc049 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.128650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:36.150244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:36.155093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.155740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.156024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.156296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] PATH_INFO: `/v3/services/network` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.156822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22c972-75e0-42c7-b081-2e9840e31475 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.156934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.158424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:36.171474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:36.172525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:36.173359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 389/765] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:35 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:36.178352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:36.226719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.227736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.228015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.228287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.228964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:36.229261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.234061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.259811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 377/766] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:36.265833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:36.398301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb493-2e59-4a2f-9691-a41bee6979f5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.434021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f9ab4-38a0-4135-908a-516eddec587b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.446885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.447550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.447834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.448105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.455685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.457577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:36.470736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:36.471654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:36.472539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 378/767] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:36.479936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:36.503740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:36.505968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe78271-204a-42a4-b1c8-77b7efad9cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.506835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe78271-204a-42a4-b1c8-77b7efad9cbb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.507327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe78271-204a-42a4-b1c8-77b7efad9cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.507794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe78271-204a-42a4-b1c8-77b7efad9cbb None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.508618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe78271-204a-42a4-b1c8-77b7efad9cbb None admin] RBAC: Adding query filter params (name=network) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:36.509128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe78271-204a-42a4-b1c8-77b7efad9cbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.510745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe78271-204a-42a4-b1c8-77b7efad9cbb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.515953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 390/768] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:36.517322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0a1f-5379-4a8c-bf92-38333314c401 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.517887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0a1f-5379-4a8c-bf92-38333314c401 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.518535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0a1f-5379-4a8c-bf92-38333314c401 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.520115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 391/769] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:36.526200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f0a1f-5379-4a8c-bf92-38333314c401 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:36.589551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:36.612540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c18dcd-d452-4f02-beba-d0341eb465ef None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.642405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f0a1f-5379-4a8c-bf92-38333314c401 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.652419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0261a-294d-4e66-8ac6-cdbb92c840f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.653308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0261a-294d-4e66-8ac6-cdbb92c840f8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.653867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0261a-294d-4e66-8ac6-cdbb92c840f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.654384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0261a-294d-4e66-8ac6-cdbb92c840f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.655308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0261a-294d-4e66-8ac6-cdbb92c840f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:36.655916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0261a-294d-4e66-8ac6-cdbb92c840f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.657542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0261a-294d-4e66-8ac6-cdbb92c840f8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.671743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 379/770] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:36.673738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e20e1d-3917-4a79-afb3-219fa8243f0d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.674335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e20e1d-3917-4a79-afb3-219fa8243f0d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.674857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e20e1d-3917-4a79-afb3-219fa8243f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.684164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.684778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.685056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.685327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.685957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] RBAC: Adding query filter params (type=network) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:36.686257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.687648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.693828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 392/771] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:36.694780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:36.696498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-04e20e1d-3917-4a79-afb3-219fa8243f0d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:36.698066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-04e20e1d-3917-4a79-afb3-219fa8243f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:36.768787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 380/772] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:36.769957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e20e1d-3917-4a79-afb3-219fa8243f0d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:36.772307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:36.812625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7a789-a4e1-4a92-9f62-fb6fb41e9d21 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.833528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:36.851027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f415e63-33e8-443c-9977-98f28ee94417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.851618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f415e63-33e8-443c-9977-98f28ee94417 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.851891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f415e63-33e8-443c-9977-98f28ee94417 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.852157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f415e63-33e8-443c-9977-98f28ee94417 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.852868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f415e63-33e8-443c-9977-98f28ee94417 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=909708a2a44a48e7b62a6f6a8d792f11) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:36.853169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f415e63-33e8-443c-9977-98f28ee94417 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.857207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f415e63-33e8-443c-9977-98f28ee94417 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.865528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 393/773] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:36 2022] GET /identity/v3/role_assignments?user.id=909708a2a44a48e7b62a6f6a8d792f11&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:36.867783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8b6a4-c3a1-41f4-88a2-eb22ffe2c455 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.868318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8b6a4-c3a1-41f4-88a2-eb22ffe2c455 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.868804 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8b6a4-c3a1-41f4-88a2-eb22ffe2c455 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.870740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 394/774] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:36.877778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d53e-8811-4ac2-8e1e-34c2e0653402 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.883942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d53e-8811-4ac2-8e1e-34c2e0653402 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.883942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d53e-8811-4ac2-8e1e-34c2e0653402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.911045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e20e1d-3917-4a79-afb3-219fa8243f0d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:36.919789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-27a7d53e-8811-4ac2-8e1e-34c2e0653402 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:36.921037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-27a7d53e-8811-4ac2-8e1e-34c2e0653402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:36.953572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:36.954209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:36.954489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:36.954792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:36.955719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] RBAC: Adding query filter params (interface=public, service_id=4f000fe1db134c34a3415746eee1c47e, region_id=RegionOne) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:36.956029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:36.957640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:36.967800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 381/775] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:12:36 2022] GET /identity/v3/endpoints?service_id=4f000fe1db134c34a3415746eee1c47e&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:36.972901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:37.069934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 395/776] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:37.070829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7d53e-8811-4ac2-8e1e-34c2e0653402 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:37.099376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:37.169320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b19d0-b400-4db3-838d-300079bfd2ff None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:37.227578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7d53e-8811-4ac2-8e1e-34c2e0653402 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:37.243537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:37.244148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.244453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.244721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.245325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:37.246708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:37.256601 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 382/777] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:12:36 2022] GET /identity/v3/services => generated 772 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:37.265986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:37.301587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:37.302229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.302502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.302796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.323796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:37.330565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:37.331584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:37.332435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 396/778] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:37.337760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:37.459079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:37.481459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5529d73-354f-4258-8749-60bb0285d81a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:37.515368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:37.522252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:37.523840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.524123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.524396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.525097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:37.525396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:37.527565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:37.541463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 383/779] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:37.548330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:37.667560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d722b-cb81-4cee-a605-bd9ba20ff266 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:37.707948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:37.708726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.709012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.709280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.721342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:37.722800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:37.741867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:37.742983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:37.743876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:37.743876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:37.743876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:37.743876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 384/780] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:37.749192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:37.801662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:37.809176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e2787-e146-4e8c-958c-9b99df836e24 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:37.864249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1975e-22d7-4754-8262-57470affb91d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:37.898087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43e3b1-e062-42e2-92a1-29bd6d14ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:37.899042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43e3b1-e062-42e2-92a1-29bd6d14ff0c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.899318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43e3b1-e062-42e2-92a1-29bd6d14ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.899585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43e3b1-e062-42e2-92a1-29bd6d14ff0c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.900168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43e3b1-e062-42e2-92a1-29bd6d14ff0c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:37.902199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43e3b1-e062-42e2-92a1-29bd6d14ff0c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:37.902900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7a9db-c550-4ffb-a4f0-018ec7a43828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:37.903577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7a9db-c550-4ffb-a4f0-018ec7a43828 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.903924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7a9db-c550-4ffb-a4f0-018ec7a43828 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.904371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7a9db-c550-4ffb-a4f0-018ec7a43828 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.905317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7a9db-c550-4ffb-a4f0-018ec7a43828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:37.905620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7a9db-c550-4ffb-a4f0-018ec7a43828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:37.907330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7a9db-c550-4ffb-a4f0-018ec7a43828 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2b43e3b1-e062-42e2-92a1-29bd6d14ff0c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:37.912268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:37.914397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:37.915557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 397/781] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 24 12:12:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:37.922686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23f020-fbc7-4238-a6d2-47d5c549e9ab None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.922686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23f020-fbc7-4238-a6d2-47d5c549e9ab None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.922686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23f020-fbc7-4238-a6d2-47d5c549e9ab None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.922686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 398/782] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:37.922686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23f020-fbc7-4238-a6d2-47d5c549e9ab None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:37.930160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 385/783] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:37.939462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3824f52-fad7-47fc-a6f1-a3efbd83feb4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:37.939462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3824f52-fad7-47fc-a6f1-a3efbd83feb4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:37.939462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3824f52-fad7-47fc-a6f1-a3efbd83feb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:37.955788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e3824f52-fad7-47fc-a6f1-a3efbd83feb4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:37.957001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e3824f52-fad7-47fc-a6f1-a3efbd83feb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:37.974108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:38.034375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 386/784] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:38.035519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3824f52-fad7-47fc-a6f1-a3efbd83feb4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.048435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23f020-fbc7-4238-a6d2-47d5c549e9ab None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:38.073115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:38.136577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:38.137488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:38.137783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:38.138150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:38.138949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:38.139353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:38.141191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:38.151419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3824f52-fad7-47fc-a6f1-a3efbd83feb4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:38.156390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 399/785] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 12:12:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:38.164185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.195755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:38.196651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:38.197071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:38.198286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:38.206740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:38.208611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:38.222791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:38.223851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:12:38.224861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:38.227343 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 387/786] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:38.233655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.339021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:38.343531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd594a-7c78-4971-b836-c7dc27e175e2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:38.379886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:38.380740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:38.381162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:38.381573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:38.382472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:38.382942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:38.385353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:38.406691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 388/787] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:38.407747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.444627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07e991-720f-4a70-8980-819c6f6ec300 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:38.463554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:38.490395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:38.491427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:38.492231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:38.492622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:38.493646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:38.494123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:38.496532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:38.506688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 400/788] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:38.506688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.546882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94b44-76bb-47d1-8ce2-8b501c50c4ac None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:38.584703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:38.585215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:38.586050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:38.586316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:38.598188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:38.603069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:38.604816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:38.605655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 389/789] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:38.606421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.623645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:38.643039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42fdb7-a757-48f8-84fd-f0d28918ce77 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:38.690226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:38.691243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:38.691675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:38.692099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:38.729809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bf633-66bb-4b1e-a430-26b9a68255c3 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:38.736551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:38.770807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:38.771480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:38.772486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:38.772877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:38.773770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] RBAC: Adding query filter params (interface=public, service_id=36e0c57221d54ffdb82dab94c97e653f, region_id=RegionOne) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:38.774177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:38.776343 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:38.782081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:38.782217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 390/790] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/endpoints?service_id=36e0c57221d54ffdb82dab94c97e653f&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:38.783281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.783991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:38.902441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 401/791] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:12:38.903632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:38.983630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:38.993799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:39.018918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbf321-6a90-4a46-bcb9-742af5b52c3a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.019536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f1241-f424-478e-8859-88da5c3100b1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.053642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.054168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.054274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80047f-81b7-4284-bf84-d91f6a59fa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.054487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.054728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.054827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80047f-81b7-4284-bf84-d91f6a59fa75 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.055102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80047f-81b7-4284-bf84-d91f6a59fa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.055366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80047f-81b7-4284-bf84-d91f6a59fa75 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.055461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:39.055678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.055960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80047f-81b7-4284-bf84-d91f6a59fa75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.057709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.057879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80047f-81b7-4284-bf84-d91f6a59fa75 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.065597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 391/792] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:39.066427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.068081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 402/793] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:12:38 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:39.073127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a1577-2a3a-4d6e-87dd-093e7a2a2b17 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.073442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a1577-2a3a-4d6e-87dd-093e7a2a2b17 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.073671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a1577-2a3a-4d6e-87dd-093e7a2a2b17 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.078725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 403/794] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:39.078725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a1577-2a3a-4d6e-87dd-093e7a2a2b17 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.196338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714659e-ecfe-4039-8801-cbf4d67d8720 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.215037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a1577-2a3a-4d6e-87dd-093e7a2a2b17 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.239448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b78dd-65b8-40e5-8774-192de1e1b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.240177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b78dd-65b8-40e5-8774-192de1e1b7ba None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.240972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b78dd-65b8-40e5-8774-192de1e1b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.240972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b78dd-65b8-40e5-8774-192de1e1b7ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.250542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:39.253889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:39.257345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b78dd-65b8-40e5-8774-192de1e1b7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.259171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b78dd-65b8-40e5-8774-192de1e1b7ba None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.272010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.272780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.273159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.273532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-4d5b78dd-65b8-40e5-8774-192de1e1b7ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:39.273965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:39.275137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:39.276097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:39.276097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:39.276097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:39.276097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 392/795] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:39.277767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9bab9-90b5-4d05-a306-9b9fc9d5fdfa None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.277767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9bab9-90b5-4d05-a306-9b9fc9d5fdfa None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.277767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9bab9-90b5-4d05-a306-9b9fc9d5fdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.293885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.295559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.295882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-add9bab9-90b5-4d05-a306-9b9fc9d5fdfa None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:39.297136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-add9bab9-90b5-4d05-a306-9b9fc9d5fdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:39.316168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:39.317382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:39.318531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:39.318531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:39.318531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:39.319986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 404/796] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:39.319986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.384306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 393/797] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:39.391549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9bab9-90b5-4d05-a306-9b9fc9d5fdfa None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.444468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3af50b-3840-4cc1-a7d8-f673af44b4fe None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.480939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.481992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.482440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.482839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.483685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:39.484090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.485544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.503627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 405/798] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:39.504870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.588438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:39.597049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9bab9-90b5-4d05-a306-9b9fc9d5fdfa None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.623294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:39.642900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24926fac-e6e3-4015-abf0-090617aabfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.643873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24926fac-e6e3-4015-abf0-090617aabfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.644258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24926fac-e6e3-4015-abf0-090617aabfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.644631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24926fac-e6e3-4015-abf0-090617aabfe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.645635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24926fac-e6e3-4015-abf0-090617aabfe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:39.646040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24926fac-e6e3-4015-abf0-090617aabfe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.649146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24926fac-e6e3-4015-abf0-090617aabfe5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.656150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8033b02-6ca2-4ffb-aa4a-f72cecf56e1e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.663779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 394/799] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:39.665243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6b9b2-b36e-40fa-9d5d-2dd1e7ded2ad None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.666777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6b9b2-b36e-40fa-9d5d-2dd1e7ded2ad None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.667223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6b9b2-b36e-40fa-9d5d-2dd1e7ded2ad None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.672521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 395/800] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:39.672521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6b9b2-b36e-40fa-9d5d-2dd1e7ded2ad None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.708198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ebfa1-36ce-4114-a75a-bd11a89c9769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.709399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ebfa1-36ce-4114-a75a-bd11a89c9769 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.709681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ebfa1-36ce-4114-a75a-bd11a89c9769 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.709949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ebfa1-36ce-4114-a75a-bd11a89c9769 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.718106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ebfa1-36ce-4114-a75a-bd11a89c9769 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.720228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ebfa1-36ce-4114-a75a-bd11a89c9769 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-536ebfa1-36ce-4114-a75a-bd11a89c9769 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:39.732046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:39.732936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:39.733915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 406/801] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:39.734393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b196ae-4472-439f-8b7b-2d9d6762ba43 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.735961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b196ae-4472-439f-8b7b-2d9d6762ba43 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.735961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b196ae-4472-439f-8b7b-2d9d6762ba43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.752766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-33b196ae-4472-439f-8b7b-2d9d6762ba43 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:39.753938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-33b196ae-4472-439f-8b7b-2d9d6762ba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:39.785211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6b9b2-b36e-40fa-9d5d-2dd1e7ded2ad None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.819922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 407/802] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:39.820747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b196ae-4472-439f-8b7b-2d9d6762ba43 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.832631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.833495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.833960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.834420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.844608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:39.846510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:39.859968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:39.860887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:39.861828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:39.864359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 396/803] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:39.865361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:39.867283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:39.917652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:39.946111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b196ae-4472-439f-8b7b-2d9d6762ba43 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.982142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:39.982784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:39.983061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:39.983328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:39.993679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:39.996683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6beae-1b14-4859-88fd-ea6eaeda108c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:39.998384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:39.999350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:40.000192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 408/804] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:40.000904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.037620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.038558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.039057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.039552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.040454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.040949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.043461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.061482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 397/805] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.062639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.094294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:40.109008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b3115-5bb7-40f9-a36d-12b9daed51cf None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.140671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.141271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.141543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.141811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.142535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.142989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.144482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.162103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 409/806] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 12:12:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.163028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.171263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:40.174925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642f3fd-d8db-4f1b-9ea4-7375fa733e4c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.210412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.211342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.211661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.212092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.213754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.214177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.215685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.226872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 398/807] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.227735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.276846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96185a-5e9e-4bfd-ba78-923eb7a2bdf7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.311164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.311760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.312040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.312311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.313035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=0ed16fea638a45938eee8cdf234333f1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.313341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.314793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.320397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 410/808] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/role_assignments?user.id=0ed16fea638a45938eee8cdf234333f1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.321164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.346104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e3ef6-2148-4e19-8387-01a8475c566b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.351340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:40.384161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.385613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.386064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.386502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] PATH_INFO: `/v3/services/network` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.387135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.388598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:40.398283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:40.399387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:12:40.400252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:40.403068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 399/809] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:40.404081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.434377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:40.479942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e732c5-fa2e-431e-bfd1-dd46f2d6a764 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.524084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa33cd-514c-478b-97d6-ccf77211241e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.524375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.525118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.525394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.526431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.541927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.543903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:40.562773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:40.563823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:40.564713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:40.564713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:40.564713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:40.564713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 411/810] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:40.564713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.574792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.576607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.576886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.577157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.592579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.594062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:40.614096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:40.615193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:40.616592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:40.616592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:40.616592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:40.616592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 400/811] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:40.616840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.652350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:40.695573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce02f-6b4a-444e-a57e-4bd37bdd78d2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.711205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:40.723566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab416e4-5618-424b-810f-1ebbfc741583 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.743857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.744522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.744800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.745075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.745709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] RBAC: Adding query filter params (name=network) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.746009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.747924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.754710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 412/812] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.755788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.762442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.763620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.764115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.764591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.765567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.766062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.767886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.782197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 401/813] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.783455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.868241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae474ce-bd6d-4723-8930-83447fbbf54a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.891335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0cf02-f3e0-4d19-8082-b3f0480fb453 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:40.903382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:40.912354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.913260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.913744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.914383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.915532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.916125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.917806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.934865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 413/814] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.936321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.939243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:40.939851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:40.940207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:40.940488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:40.941910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] RBAC: Adding query filter params (type=network) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:40.941910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:40.942936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:40.953719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 402/815] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:40.953719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:40.954456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:41.054864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a243fb3-07a7-4e2c-8eb5-c6ea2bbf3c14 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.085775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f66ec-b192-48af-ae2d-e29e8618fe25 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.095315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.095937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.096224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.096497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.103392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.104855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:41.116079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:41.116954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:41.117843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 414/816] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:41.118326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.128238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc72eb-7851-4314-93d1-aa7314b43b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.129088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc72eb-7851-4314-93d1-aa7314b43b65 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.129672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc72eb-7851-4314-93d1-aa7314b43b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.130119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc72eb-7851-4314-93d1-aa7314b43b65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.138963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc72eb-7851-4314-93d1-aa7314b43b65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.140641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc72eb-7851-4314-93d1-aa7314b43b65 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-2bcc72eb-7851-4314-93d1-aa7314b43b65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:41.152666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:41.153621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:41.154572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:41.156362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 403/817] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:41.157623 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e486-90fe-431c-8451-dfd6a5b10b51 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.158059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e486-90fe-431c-8451-dfd6a5b10b51 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.158484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948e486-90fe-431c-8451-dfd6a5b10b51 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.160120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 404/818] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:41.160965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948e486-90fe-431c-8451-dfd6a5b10b51 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.175239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:41.225866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf51b3-36e2-4613-92fb-54bee3ca4454 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.227760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:41.262491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ea33d-f767-4a52-b410-4faf44b5ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.263114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ea33d-f767-4a52-b410-4faf44b5ff12 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.263387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ea33d-f767-4a52-b410-4faf44b5ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.263657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ea33d-f767-4a52-b410-4faf44b5ff12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.264310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ea33d-f767-4a52-b410-4faf44b5ff12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.265662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ea33d-f767-4a52-b410-4faf44b5ff12 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.268712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948e486-90fe-431c-8451-dfd6a5b10b51 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.302463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.303096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.303377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.303650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.304339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:41.304645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.306062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.318857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 405/819] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:41.319661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.390687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 415/820] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:12:41 2022] POST /identity/v3/endpoints => generated 337 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:41.390687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d183c8f-0244-42ca-992c-e8240fc3451b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.390687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d183c8f-0244-42ca-992c-e8240fc3451b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.390687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d183c8f-0244-42ca-992c-e8240fc3451b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.421789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0d183c8f-0244-42ca-992c-e8240fc3451b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:41.423228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0d183c8f-0244-42ca-992c-e8240fc3451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:41.441924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e3bb3-95a8-4cdb-9d1e-b22eaa5f0f1c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.472008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:41.492818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.493597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.494049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.494351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.495119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:41.495465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.497041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.517186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 406/821] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 12:12:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:41.518197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.531705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 416/822] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:41.533393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d183c8f-0244-42ca-992c-e8240fc3451b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.578897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:41.672964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68cd9a-0b97-430c-adf7-dd66bb43a975 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.683120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d183c8f-0244-42ca-992c-e8240fc3451b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.725905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.726292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.726575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.726877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/44144b71637548c0b31368fc8114b6b6/roles/c26057e0945745a39399a115b756ea2b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.738596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.739297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.739577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.739847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.749051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=44144b71637548c0b31368fc8114b6b6, role_id=c26057e0945745a39399a115b756ea2b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.751970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.755211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.757195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:41.776026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:41.777019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:41.777955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:41.777955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:41.777955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:41.777955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 417/823] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:41.783108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.861553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 407/824] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:41 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/44144b71637548c0b31368fc8114b6b6/roles/c26057e0945745a39399a115b756ea2b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:41.862481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.888301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:41.903122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:41.905487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae22f60-34e2-4818-b169-df2f4ace9a6b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:41.946313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:41.946879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:41.947163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:41.947476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:41.948139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:41.948443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:41.949901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:41.967573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 418/825] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:41.968992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:41.983947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2be3-5dc1-4564-87d7-1793c617a947 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:42.023081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:42.024540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.027407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.027407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.027407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:42.027407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:42.043755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:42.044867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:42.045717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 408/826] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 24 12:12:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:42.051270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:42.106335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa00170-8d7d-4771-9dce-1ee630eb7763 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:42.141514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:42.142507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.143017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.143578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.151364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:42.153071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:42.158033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:42.165258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:42.166789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:42.171027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:42.171490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe4609-cbe5-4611-81a7-d8a8b561ec9d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:42.171565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 419/827] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:42.174350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:42.189527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:42.210868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:42.210868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.210868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.210868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.211255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:42.211255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:42.211255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:42.220568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 409/828] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 12:12:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:42.221681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:42.309988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715d27f-58a9-437d-9593-35bf5c8ceade None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:42.325161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984993-d6a4-4964-954f-3caccaabdbee None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:42.361377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:42.362325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.362682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.363094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.364414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:42.364736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:42.366689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:42.368777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:42.370136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.370620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.371109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.371955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:42.372476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:42.374075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:42.382683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 420/829] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:42.384058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 410/830] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 12:12:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:42.385001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:42.387222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:42.407492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:42.464672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:42.497900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2116e-245a-4514-b979-55c4e59f6a7e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:42.500087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90d2f1-956e-4b10-b50d-366a57584827 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:42.536388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8804ce8-a813-4687-b43d-900ff7cef482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:42.537042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8804ce8-a813-4687-b43d-900ff7cef482 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.537319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8804ce8-a813-4687-b43d-900ff7cef482 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.537595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8804ce8-a813-4687-b43d-900ff7cef482 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/909708a2a44a48e7b62a6f6a8d792f11/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.539545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8804ce8-a813-4687-b43d-900ff7cef482 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=909708a2a44a48e7b62a6f6a8d792f11, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:42.540997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8804ce8-a813-4687-b43d-900ff7cef482 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:42.542163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:42.542977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.543445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.543903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.544693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:42.546296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:42.605225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 411/831] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:42 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/909708a2a44a48e7b62a6f6a8d792f11/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:42.606482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb609-3957-45de-a425-e0107ba4e8a1 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.607058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb609-3957-45de-a425-e0107ba4e8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.607328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb609-3957-45de-a425-e0107ba4e8a1 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.610694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 412/832] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:42.614471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621e5f6-5487-45e8-a7c1-a0d064ee7c18 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:42.614850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621e5f6-5487-45e8-a7c1-a0d064ee7c18 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:42.615125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621e5f6-5487-45e8-a7c1-a0d064ee7c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:42.635379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7621e5f6-5487-45e8-a7c1-a0d064ee7c18 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:42.641410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7621e5f6-5487-45e8-a7c1-a0d064ee7c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:42.642345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 421/833] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:12:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:42.724259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 413/834] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:42.992469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:43.098137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f34d8-efa7-4bb8-8e9e-8bd65cd5c2e4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:43.123101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:43.134024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:43.135355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:43.135355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:43.135355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:43.135814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:43.137310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:43.145923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:43.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:43.147988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 422/835] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 24 12:12:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:43.153471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621e5f6-5487-45e8-a7c1-a0d064ee7c18 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:43.156746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:43.260483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621e5f6-5487-45e8-a7c1-a0d064ee7c18 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:43.295124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc03fd4-b827-4c83-b1f8-36935e1d79bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:43.295733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc03fd4-b827-4c83-b1f8-36935e1d79bc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:43.296011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc03fd4-b827-4c83-b1f8-36935e1d79bc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:43.296283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc03fd4-b827-4c83-b1f8-36935e1d79bc None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:43.296917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc03fd4-b827-4c83-b1f8-36935e1d79bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:43.297217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc03fd4-b827-4c83-b1f8-36935e1d79bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:43.298582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc03fd4-b827-4c83-b1f8-36935e1d79bc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:43.303637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 414/836] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:43.309754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:43.414485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b79d0-de9b-4b89-aefb-78d5fe5880eb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:43.450312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaabb10-ef91-4fd5-8bdf-b3ea85bebbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:43.450941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaabb10-ef91-4fd5-8bdf-b3ea85bebbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:43.451219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaabb10-ef91-4fd5-8bdf-b3ea85bebbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:43.451489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaabb10-ef91-4fd5-8bdf-b3ea85bebbf1 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:43.452117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaabb10-ef91-4fd5-8bdf-b3ea85bebbf1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:43.452442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaabb10-ef91-4fd5-8bdf-b3ea85bebbf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:43.453798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaabb10-ef91-4fd5-8bdf-b3ea85bebbf1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:43.458832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 423/837] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:43.833651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adffc5-745c-42ea-b2ff-1086a38b942f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:43.834054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adffc5-745c-42ea-b2ff-1086a38b942f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:43.834229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adffc5-745c-42ea-b2ff-1086a38b942f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:43.835852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 415/838] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:43.841629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c07c-c2ba-41b4-94bb-b6b8343063f6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:43.842077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c07c-c2ba-41b4-94bb-b6b8343063f6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:43.842167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9c07c-c2ba-41b4-94bb-b6b8343063f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:43.862073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7df9c07c-c2ba-41b4-94bb-b6b8343063f6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:43.863305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7df9c07c-c2ba-41b4-94bb-b6b8343063f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:43.912880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:43.937790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 424/839] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:44.202108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adffc5-745c-42ea-b2ff-1086a38b942f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:44.271134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:44.305672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adffc5-745c-42ea-b2ff-1086a38b942f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:44.341016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:44.341643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.341917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.342190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.352499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:44.357574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:44.358501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:44.359359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 416/840] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:12:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:44.364670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c07c-c2ba-41b4-94bb-b6b8343063f6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:44.471839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9c07c-c2ba-41b4-94bb-b6b8343063f6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:44.509140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3a58-153d-4e63-b82b-a74abc52824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:44.510533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3a58-153d-4e63-b82b-a74abc52824a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.511016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3a58-153d-4e63-b82b-a74abc52824a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.511297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3a58-153d-4e63-b82b-a74abc52824a None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.512025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e3a58-153d-4e63-b82b-a74abc52824a None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:44.512331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e3a58-153d-4e63-b82b-a74abc52824a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:44.513700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e3a58-153d-4e63-b82b-a74abc52824a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:44.526765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 425/841] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:12:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:44.528383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:44.599826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343505b1-0f4e-475d-a081-1aa25969c829 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.600167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343505b1-0f4e-475d-a081-1aa25969c829 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.600382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343505b1-0f4e-475d-a081-1aa25969c829 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.601859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 426/842] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:44.608842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741902f2-c2d0-4afd-bdb5-b3d18607f686 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.610227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741902f2-c2d0-4afd-bdb5-b3d18607f686 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.610757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741902f2-c2d0-4afd-bdb5-b3d18607f686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.633131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-741902f2-c2d0-4afd-bdb5-b3d18607f686 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:44.635729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-741902f2-c2d0-4afd-bdb5-b3d18607f686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:44.640892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976abeb-9a0e-4db3-b16a-f332ae0cb8ef None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:44.670112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:44.678644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005829c0-286b-4c63-bf34-a53c40fa9924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:44.684747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005829c0-286b-4c63-bf34-a53c40fa9924 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.684747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005829c0-286b-4c63-bf34-a53c40fa9924 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.684747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005829c0-286b-4c63-bf34-a53c40fa9924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.685062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:44.690742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005829c0-286b-4c63-bf34-a53c40fa9924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:44.692181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005829c0-286b-4c63-bf34-a53c40fa9924 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-005829c0-286b-4c63-bf34-a53c40fa9924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:44.714690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:44.715695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:44.716621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:44.716621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:44.716621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:44.716793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 427/843] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:44.716793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789c4a6-647c-448c-a51d-1e440b343879 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.716793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789c4a6-647c-448c-a51d-1e440b343879 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.716793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789c4a6-647c-448c-a51d-1e440b343879 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.716793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 428/844] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:44.717094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 417/845] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:44.717162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789c4a6-647c-448c-a51d-1e440b343879 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:44.722964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede14c-5eb2-4a44-9520-78152a7ff718 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.722964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede14c-5eb2-4a44-9520-78152a7ff718 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.722964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ede14c-5eb2-4a44-9520-78152a7ff718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.737388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-13ede14c-5eb2-4a44-9520-78152a7ff718 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:44.738581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-13ede14c-5eb2-4a44-9520-78152a7ff718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:44.814762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 418/846] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:44.832414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789c4a6-647c-448c-a51d-1e440b343879 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:44.869873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:44.876900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:44.876900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:44.876900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:44.876900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:44.876900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:44.876900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:44.887242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 429/847] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:44.894741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ede14c-5eb2-4a44-9520-78152a7ff718 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.005745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ede14c-5eb2-4a44-9520-78152a7ff718 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.008771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.019648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.041400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.042038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.042306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.042632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.050065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.050897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.052080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:45.065121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:45.066099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:45.067080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 419/848] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:12:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:45.071973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.132566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e148b-19fd-4413-b85b-b59ab2462020 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.167558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.168179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.168733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.179179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:45.183384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dfd4-eddd-4f2a-bd00-30a5538da402 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:45.184833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:45.185737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:45.186571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 430/849] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:45.187549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.225316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.226015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.226291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.226562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.227371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:45.227688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.229663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.252326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 420/850] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:45.253191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.284803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.289546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.296929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b060-73b2-4963-939f-1d144b00c4ce None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.330579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.331234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.331519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.331794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.332378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.333780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:45.343127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:45.344217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:45.345054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 431/851] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:45.345879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.373712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf1375-2463-4258-978f-f78a6da862fb None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.411573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.412176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.412459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.412925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.416746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:45.416746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.416746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.425405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 421/852] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:45.425405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.469420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274d606-030a-45b9-9784-3cb86130ba2e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.503303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325ae13-c8d0-4f06-a1d1-3b4da9638dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.503894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ae13-c8d0-4f06-a1d1-3b4da9638dab None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.504166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ae13-c8d0-4f06-a1d1-3b4da9638dab None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.504449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ae13-c8d0-4f06-a1d1-3b4da9638dab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.519313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.528920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d805b-be54-40f4-a44d-d4c22ba1577e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.532822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.563218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.563745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.564023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.564345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.564973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:45.565280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.566642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.571383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 422/853] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:45.572876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.574855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325ae13-c8d0-4f06-a1d1-3b4da9638dab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.576283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325ae13-c8d0-4f06-a1d1-3b4da9638dab None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.671976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b7b78-f9d2-4697-81b2-9ef448e197bc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.673449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 432/854] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:12:45.674705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b64f-7c3e-4487-84ad-e977063b590a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.674997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b64f-7c3e-4487-84ad-e977063b590a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.675268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b64f-7c3e-4487-84ad-e977063b590a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.676622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 433/855] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:45.679411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b64f-7c3e-4487-84ad-e977063b590a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.707095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.707736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.708010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.708276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.719151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.720567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:45.735236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:45.736235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:45.737124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:45.737124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:45.737124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:45.737124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 423/856] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:45.737124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.740236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.745675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:45.782268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b64f-7c3e-4487-84ad-e977063b590a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.816554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51d4f0-9fe1-433d-8cf3-23de18a3bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.817082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51d4f0-9fe1-433d-8cf3-23de18a3bfae None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.817362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51d4f0-9fe1-433d-8cf3-23de18a3bfae None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.817636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51d4f0-9fe1-433d-8cf3-23de18a3bfae None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.818256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51d4f0-9fe1-433d-8cf3-23de18a3bfae None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:45.818569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51d4f0-9fe1-433d-8cf3-23de18a3bfae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.820174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51d4f0-9fe1-433d-8cf3-23de18a3bfae None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.824801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 434/857] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:45.825911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5ca70-4276-4147-a07d-85a6baa5af38 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.826194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5ca70-4276-4147-a07d-85a6baa5af38 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.826466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5ca70-4276-4147-a07d-85a6baa5af38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.834395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf2524-4906-4ecd-8d64-5a867c9e5127 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:45.845515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a1b5ca70-4276-4147-a07d-85a6baa5af38 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:45.846777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a1b5ca70-4276-4147-a07d-85a6baa5af38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:45.882168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:45.883134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:45.883134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:45.883375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:45.895634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:45.897130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:45.916598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:45.917627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:45.923446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:45.923446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:45.923446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:45.923446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 424/858] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:45.923446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.943098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 435/859] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:45.943098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5ca70-4276-4147-a07d-85a6baa5af38 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:45.999126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.010550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.044221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e21b13-0b8a-4305-9a38-b5eb96ef50e0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.059510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5ca70-4276-4147-a07d-85a6baa5af38 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.080870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.081398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.081677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.081949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.082782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:46.083086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.084550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.094558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.095188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.095472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.095743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.096377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:46.096685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.098040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.102971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 425/860] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:46.104439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:46.110422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 436/861] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:46.120322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:46.213395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ef46c-c882-45df-ade3-1050d345fb77 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.225360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a54b50-540d-4037-a5dd-e669ce289795 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.232843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.240501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.249468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.250144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.250419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.250710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.257699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.259175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.262812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.263420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.263699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.263972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.270757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.272236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:46.272817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:46.273953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 426/862] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:12:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:46.275020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:46.298125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:46.299318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:46.300198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 437/863] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:46.300682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:46.386374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa746f-12cb-4499-8156-c709315f3eb5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.414538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268d8a-284d-47f0-a893-b91940230139 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.421385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.422048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.425015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.425015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.425015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.425015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.452175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.452768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.453041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.453307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.454021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:46.454319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.455763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.473232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 438/864] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 12:12:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:46.474058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:46.500245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 427/865] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:12:46 2022] POST /identity/v3/services => generated 269 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:46.501534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:46.505621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.569984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.615732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e815f-8a9f-4d25-837c-e0be4477530a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.670989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03af54c-5c5e-4393-816b-5dc336729ef5 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.672875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.673486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.673762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.674033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.674754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:46.675061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.676486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.689661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 439/866] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:46.695868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:46.711247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.713328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.713328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.713328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.783250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.784695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.814227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.830626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:46.830827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98e0a2-cf81-4f42-98aa-5f063ec98938 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:46.868503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:46.869822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:46.870170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:46.870534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:46.871361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=44144b71637548c0b31368fc8114b6b6) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:46.871739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:46.873236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:46.885144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 440/867] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:46 2022] GET /identity/v3/role_assignments?user.id=44144b71637548c0b31368fc8114b6b6&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:46.920674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 428/868] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:46 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:12:46.926900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:47.043383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95a1fe-c5ee-4fad-ba51-f33252a5f6d5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:47.080649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:47.081588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:47.082043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:47.082487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:47.092983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:47.094671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:47.110590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:47.111692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:47.112564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:47.112564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:47.112564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:47.113128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 441/869] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:47.117738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:47.231012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa204-d1ca-4098-acd8-4003aeda020e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:47.266897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:47.267514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:47.267766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:47.268042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:47.268683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:47.268986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:47.270349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:47.271772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:47.282950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 429/870] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:47.288711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:47.292122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:47.393381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6189eb2-ba78-464e-a65c-7a708f4f4a7d None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:47.429407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:47.429694 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:47.434668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:47.434668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:47.437618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:47.439157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:47.450444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:47.451383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:47.452276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:47.456604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 442/871] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:47.457635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:47.565070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab601c-7729-46bb-b68f-1ca95b2274e8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:47.600226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17055ba7-620d-4ddd-bacb-fb5d0770265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:47.600825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055ba7-620d-4ddd-bacb-fb5d0770265e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:47.601101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055ba7-620d-4ddd-bacb-fb5d0770265e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:47.601370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055ba7-620d-4ddd-bacb-fb5d0770265e None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:47.602053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17055ba7-620d-4ddd-bacb-fb5d0770265e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:47.602353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17055ba7-620d-4ddd-bacb-fb5d0770265e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:47.604365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17055ba7-620d-4ddd-bacb-fb5d0770265e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:47.617387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 430/872] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:47.623482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:47.688449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:47.728697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51090340-de35-4acf-88f7-4cd013f2f628 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:47.764360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39984ba4-f63b-41c3-b5cc-13564b337d24 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:47.764703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39984ba4-f63b-41c3-b5cc-13564b337d24 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:47.764905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39984ba4-f63b-41c3-b5cc-13564b337d24 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:47.766399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 431/873] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:12:47.767031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:47.767655 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:47.767975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:47.768282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:47.768995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=909708a2a44a48e7b62a6f6a8d792f11) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:47.769292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:47.770719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:47.773008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2781d9-7b9e-4ff9-b6e8-82aeec1ef3a9 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:47.773284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2781d9-7b9e-4ff9-b6e8-82aeec1ef3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:47.773608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2781d9-7b9e-4ff9-b6e8-82aeec1ef3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:47.780693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 443/874] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:47 2022] GET /identity/v3/role_assignments?user.id=909708a2a44a48e7b62a6f6a8d792f11&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:47.799168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-fe2781d9-7b9e-4ff9-b6e8-82aeec1ef3a9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:47.800806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fe2781d9-7b9e-4ff9-b6e8-82aeec1ef3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:47.858939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:47.878199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 432/875] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:47 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:48.146819 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:48.247353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268a502-12b4-478f-8f5f-4e69e4d1d933 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:48.274715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:48.287361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3917b2-07d1-4abe-97e9-f171ad642e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:48.288366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3917b2-07d1-4abe-97e9-f171ad642e98 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:48.293199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3917b2-07d1-4abe-97e9-f171ad642e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:48.293199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3917b2-07d1-4abe-97e9-f171ad642e98 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:48.293199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3917b2-07d1-4abe-97e9-f171ad642e98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:48.293199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3917b2-07d1-4abe-97e9-f171ad642e98 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:48.406825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 444/876] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:12:48 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:48.549909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42feb9-8e31-4158-ba01-3b9b3a9514dd None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:48.550422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42feb9-8e31-4158-ba01-3b9b3a9514dd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:48.550422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42feb9-8e31-4158-ba01-3b9b3a9514dd None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:48.551785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 433/877] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:48.557345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e32133-be71-4872-9ea5-8a66c7a89ad3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:48.557635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e32133-be71-4872-9ea5-8a66c7a89ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:48.558090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e32133-be71-4872-9ea5-8a66c7a89ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:48.580914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d3e32133-be71-4872-9ea5-8a66c7a89ad3 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:48.582109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d3e32133-be71-4872-9ea5-8a66c7a89ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:48.662738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 445/878] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:48.936382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42feb9-8e31-4158-ba01-3b9b3a9514dd None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:48.966075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29bb45-286a-4fd6-bf15-dd2e3359c653 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:48.966420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29bb45-286a-4fd6-bf15-dd2e3359c653 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:48.966632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29bb45-286a-4fd6-bf15-dd2e3359c653 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:48.968310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 446/879] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:48.973258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2a542-98a0-43ec-b464-3f862199c0f6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:48.973619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2a542-98a0-43ec-b464-3f862199c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:48.973814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2a542-98a0-43ec-b464-3f862199c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:48.995409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-93f2a542-98a0-43ec-b464-3f862199c0f6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:48.996877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-93f2a542-98a0-43ec-b464-3f862199c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:49.053079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42feb9-8e31-4158-ba01-3b9b3a9514dd None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.086141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 447/880] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:49.092258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.092886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.093168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.093438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.094024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:49.095469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:49.105185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:49.106114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:49.107323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 434/881] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 24 12:12:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:49.112637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2a542-98a0-43ec-b464-3f862199c0f6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:49.156058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:49.213982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2a542-98a0-43ec-b464-3f862199c0f6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.249093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.249699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.249977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.250249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.250980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:49.251281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:49.252697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:49.257559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 448/882] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:49.263877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:49.314386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:49.371607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456490fa-c17f-4f1a-92a1-295f7c3408fc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.405843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:49.406766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.407379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.407697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.407979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.408798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:49.409105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:49.411375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:49.416926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 435/883] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:49.423502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:49.530717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de193310-cead-44d4-b8f7-7033bd225151 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.533934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3da1aa-0378-4ab2-add2-5e7d82dc0002 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.542679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:49.544398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:49.567248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.567924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.568195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.568475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.569205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] RBAC: Adding query filter params (interface=public, service_id=4a8a16c71711419fb6062fb2f2b47835, region_id=RegionOne) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:49.569514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:49.571563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:49.581639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.581853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 436/884] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:12:49 2022] GET /identity/v3/endpoints?service_id=4a8a16c71711419fb6062fb2f2b47835&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:49.581973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.581973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.581973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.583157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:49.598254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:49.605802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:49.607044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:49.607906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 449/885] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 12:12:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:49.620278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:49.715253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403063-d22d-4a38-9fbb-8a1dd4dfb5e2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.747874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a616303-23c0-438e-9e64-c77b71454ddb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.753849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.754456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.754804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.755081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.755685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:49.757052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:49.770466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 437/886] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:12:49 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:12:49.788102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a036aad-1246-42ed-8ec6-d6860dded4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.789098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036aad-1246-42ed-8ec6-d6860dded4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.789575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036aad-1246-42ed-8ec6-d6860dded4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.790046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a036aad-1246-42ed-8ec6-d6860dded4c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.791044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a036aad-1246-42ed-8ec6-d6860dded4c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:49.791580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a036aad-1246-42ed-8ec6-d6860dded4c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:49.793216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a036aad-1246-42ed-8ec6-d6860dded4c1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:49.803407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 450/887] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 12:12:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:49.809406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:49.813339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5b9c7-2602-44b3-8d2f-75d324d7ed11 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.814555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5b9c7-2602-44b3-8d2f-75d324d7ed11 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.814859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5b9c7-2602-44b3-8d2f-75d324d7ed11 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.816350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 451/888] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:12:49.822965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e3081-cddb-4f44-bdd3-8b95966de1cc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.823283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e3081-cddb-4f44-bdd3-8b95966de1cc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.823527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e3081-cddb-4f44-bdd3-8b95966de1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:49.848277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1e9e3081-cddb-4f44-bdd3-8b95966de1cc None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:49.850361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1e9e3081-cddb-4f44-bdd3-8b95966de1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:49.864956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:49.885173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:49.948477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 452/889] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:49.951964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946964f-9908-4136-bac2-de306c0bcc46 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:49.987708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:49.988340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:49.988617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:49.988891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.000233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.001673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:50.016020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:50.016949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:50.017830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:50.017830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:50.017830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:50.017830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 438/890] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:50.023314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e3081-cddb-4f44-bdd3-8b95966de1cc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.128012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e3081-cddb-4f44-bdd3-8b95966de1cc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:50.162837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:50.163453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.163759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.164037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.164683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:50.165033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.166972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.178683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 453/891] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:50.184297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.245035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.265812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:50.287128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:50.296295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e48208-06da-4fe9-9f47-ab0a1c74d8fc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:50.333064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:50.333801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.334089 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.334366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.343452 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.344989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.355686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b8951-8b22-4e53-a70d-96678ddd002c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:50.357506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:50.358437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:50.359386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 439/892] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:50.368382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.402839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:50.403442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.403724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.403997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.404783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.406776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:50.417011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:50.418056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:50.418924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:50.423904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 454/893] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:50.424459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.496083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac95546-74fd-4cb5-8c5a-478d145f7b93 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:50.512925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:50.529815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b819f-386c-41f4-9918-51888c8519d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:50.530412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b819f-386c-41f4-9918-51888c8519d5 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.530712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b819f-386c-41f4-9918-51888c8519d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.530987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b819f-386c-41f4-9918-51888c8519d5 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.531555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f9356-4ce2-4826-9555-48ea62af7b9b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:50.531736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b819f-386c-41f4-9918-51888c8519d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:50.532005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b819f-386c-41f4-9918-51888c8519d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.533426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b819f-386c-41f4-9918-51888c8519d5 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.555320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 440/894] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:50.555320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9b3df-fb2d-4aee-8fe3-20aa0d2b817f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.555320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9b3df-fb2d-4aee-8fe3-20aa0d2b817f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.555320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9b3df-fb2d-4aee-8fe3-20aa0d2b817f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.555320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 441/895] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:50.558926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9b3df-fb2d-4aee-8fe3-20aa0d2b817f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.570406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:50.572345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a28f-451e-4061-b6e1-d7e0facacb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:50.572942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a28f-451e-4061-b6e1-d7e0facacb30 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.573214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a28f-451e-4061-b6e1-d7e0facacb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.573480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a28f-451e-4061-b6e1-d7e0facacb30 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.574103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3a28f-451e-4061-b6e1-d7e0facacb30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:50.574463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3a28f-451e-4061-b6e1-d7e0facacb30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.575853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3a28f-451e-4061-b6e1-d7e0facacb30 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.585019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 455/896] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:50.585019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fb981-15e6-4f0b-b385-e6ca2b902cd3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.585019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fb981-15e6-4f0b-b385-e6ca2b902cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.585019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fb981-15e6-4f0b-b385-e6ca2b902cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.605952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-456fb981-15e6-4f0b-b385-e6ca2b902cd3 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:50.607595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-456fb981-15e6-4f0b-b385-e6ca2b902cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:50.676382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 456/897] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:50.677261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fb981-15e6-4f0b-b385-e6ca2b902cd3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.695046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9b3df-fb2d-4aee-8fe3-20aa0d2b817f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:50.744337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:50.745174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.745604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.746039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.760572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:50.785593 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fb981-15e6-4f0b-b385-e6ca2b902cd3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:50.827911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:50.828663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:50.829200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:50.830774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:50.832014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:50.832719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.834300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.839109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:50.840804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:50.846894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 457/898] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:50.859050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:50.947631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:50.964294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 442/899] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:12:50.971229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.079719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac0a4c-c875-41a4-89bd-071dd23435a0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.091103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483a6e2-3420-4981-8417-012a294ba0fe None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.103270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:51.114978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.115607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.115887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.117020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.117020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.118185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:51.127709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:51.128705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:51.129525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 458/900] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:51.135919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.137547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.138340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.138808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.139370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.152246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.154514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.171034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:51.173745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:51.174681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:51.175617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:51.175617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:51.175617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:51.178288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 443/901] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:51.187202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.257375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c13cff-5991-4f1c-ba13-5498e875fa33 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.302039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.302958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.303730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.304355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.305156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:51.305561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.305857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa416d-a7a5-4494-b291-327b10b9c8ef None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.308014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.315376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 459/902] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:51.320097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.356056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.357302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.357807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.358432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.360145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:51.360663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.362234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.382496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 444/903] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:51.383731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.394854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:51.410304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:51.445516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614911-fe64-4edb-a039-869d8bccc53a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.495688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.497114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.497870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.498348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.499444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:51.500342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.501915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.510289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 460/904] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:51.515707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.517866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5986b7-ea26-4cd4-894f-5c9c7e52d00b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.558327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.558981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.559261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.559534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.566427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.567922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:51.579333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:51.580709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:51.581657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:51.585483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 445/905] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:51.586517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.656662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf68a-117f-4c63-b6d2-c25dfe72298a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.672295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:51.691908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6134dd-576d-4f7b-b0af-e8b88aa9590c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.695917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:51.700940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.702632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.702946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.703222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.703979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] RBAC: Adding query filter params (interface=public, service_id=ff937dc62a204103aa8c71989a801b41, region_id=RegionOne) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:51.704284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.706064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.718887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 461/906] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/endpoints?service_id=ff937dc62a204103aa8c71989a801b41&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:51.720504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.736112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13a65e-b34d-492e-a80b-583337fc85cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.737261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13a65e-b34d-492e-a80b-583337fc85cf None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.737782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13a65e-b34d-492e-a80b-583337fc85cf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.738502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13a65e-b34d-492e-a80b-583337fc85cf None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.739900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13a65e-b34d-492e-a80b-583337fc85cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:51.740681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13a65e-b34d-492e-a80b-583337fc85cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.742672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13a65e-b34d-492e-a80b-583337fc85cf None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.756263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 446/907] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:51.757438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f616c9-1518-4005-90ae-f8ed134e9d67 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.757718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f616c9-1518-4005-90ae-f8ed134e9d67 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.757988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f616c9-1518-4005-90ae-f8ed134e9d67 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.759379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 447/908] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:51.763899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f616c9-1518-4005-90ae-f8ed134e9d67 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:51.839266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcf05d-cb68-4591-99b7-56a7aea7d5b0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.870451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f616c9-1518-4005-90ae-f8ed134e9d67 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:51.873879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116f7fd-2b1d-4547-8c5e-00841f7bffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.874478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116f7fd-2b1d-4547-8c5e-00841f7bffb0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.874788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116f7fd-2b1d-4547-8c5e-00841f7bffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.875063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116f7fd-2b1d-4547-8c5e-00841f7bffb0 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.875692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116f7fd-2b1d-4547-8c5e-00841f7bffb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.877049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116f7fd-2b1d-4547-8c5e-00841f7bffb0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.890695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 462/909] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:12:51.890695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f83577-ece1-43e4-9b8b-5b90012cfc37 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.890695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f83577-ece1-43e4-9b8b-5b90012cfc37 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.890695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f83577-ece1-43e4-9b8b-5b90012cfc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.904425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-47f83577-ece1-43e4-9b8b-5b90012cfc37 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:51.905651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-47f83577-ece1-43e4-9b8b-5b90012cfc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:51.923319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:51.924079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:51.924453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:51.924816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:51.925771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] RBAC: Adding query filter params (role.id=37c9983c4e5b4734a384f21bf66503e4, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=44144b71637548c0b31368fc8114b6b6) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:51.926305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:51.927916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:51.935557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 448/910] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:12:51 2022] GET /identity/v3/role_assignments?user.id=44144b71637548c0b31368fc8114b6b6&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=37c9983c4e5b4734a384f21bf66503e4 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:51.978046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:51.992113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 463/911] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:52.264502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:52.273611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:52.368707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd63e5-7d5a-4c30-8d2b-62509f991c4a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:52.404992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:52.406000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:52.406480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:52.407121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:52.407905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:52.409468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:52.424626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:52.425617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:12:52.426498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:52.430318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 449/912] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 24 12:12:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:52.435307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f83577-ece1-43e4-9b8b-5b90012cfc37 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:52.538580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f83577-ece1-43e4-9b8b-5b90012cfc37 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:52.574861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:52.575462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:52.575734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:52.576001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:52.576623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:52.576918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:52.578270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:52.583361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 464/913] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:52.588681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:52.667295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:52.691673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906bc10-0124-4d79-9a8b-5e8dffc072bb None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:52.727258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddf34b-d3a6-471d-8d19-43803f085ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:52.727866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf34b-d3a6-471d-8d19-43803f085ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:52.728764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf34b-d3a6-471d-8d19-43803f085ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:52.729038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf34b-d3a6-471d-8d19-43803f085ba6 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:52.729746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddf34b-d3a6-471d-8d19-43803f085ba6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:52.730051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddf34b-d3a6-471d-8d19-43803f085ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:52.731442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddf34b-d3a6-471d-8d19-43803f085ba6 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:52.737022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 450/914] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:52.742716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:52.803998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:52.849492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca2857-9107-447a-a3e7-992c085dec78 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:52.884437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:52.885070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:52.885311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:52.885579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:52.886241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:52.887766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:52.889101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdff74-4e3e-4ca4-87ca-7c0995dda9e9 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:52.889389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdff74-4e3e-4ca4-87ca-7c0995dda9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:52.889668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdff74-4e3e-4ca4-87ca-7c0995dda9e9 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:52.891250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 451/915] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:52.896442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b117aa-beea-4ab8-979d-f4af720c978f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:52.896728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b117aa-beea-4ab8-979d-f4af720c978f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:52.897006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b117aa-beea-4ab8-979d-f4af720c978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:52.916122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f1b117aa-beea-4ab8-979d-f4af720c978f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:52.917313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f1b117aa-beea-4ab8-979d-f4af720c978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:52.969382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 465/916] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:12:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:12:53.010442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 452/917] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:53.283341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:53.386240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96848f57-e4d7-4e10-9c3a-16c08e04db91 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:53.397423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:53.436487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee17886-f7a5-4ae9-8afe-dc4370b9e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:53.437109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17886-f7a5-4ae9-8afe-dc4370b9e240 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:53.437366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17886-f7a5-4ae9-8afe-dc4370b9e240 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:53.437634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17886-f7a5-4ae9-8afe-dc4370b9e240 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:53.438284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee17886-f7a5-4ae9-8afe-dc4370b9e240 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:53.442686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee17886-f7a5-4ae9-8afe-dc4370b9e240 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:53.557568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 466/918] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:12:53 2022] POST /identity/v3/services => generated 241 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:53.882266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba316eae-8d3a-4b43-a5eb-32df58cc0a51 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:53.882266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba316eae-8d3a-4b43-a5eb-32df58cc0a51 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:53.882266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba316eae-8d3a-4b43-a5eb-32df58cc0a51 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:53.883154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 453/919] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:53.888586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8e6a-91db-4b01-89d9-3b1746db2f6d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:53.888877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8e6a-91db-4b01-89d9-3b1746db2f6d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:53.889154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8e6a-91db-4b01-89d9-3b1746db2f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:53.909331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b1ff8e6a-91db-4b01-89d9-3b1746db2f6d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:53.910537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b1ff8e6a-91db-4b01-89d9-3b1746db2f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:53.951086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be6499-3ec3-4852-b443-73c41b4ac0da None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:53.951703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be6499-3ec3-4852-b443-73c41b4ac0da None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:53.952206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be6499-3ec3-4852-b443-73c41b4ac0da None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:53.954800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 454/920] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:12:53.959481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091225e7-ff34-4719-b8e5-6efefe5a4425 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:53.959999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091225e7-ff34-4719-b8e5-6efefe5a4425 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:53.960486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091225e7-ff34-4719-b8e5-6efefe5a4425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:53.979375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-091225e7-ff34-4719-b8e5-6efefe5a4425 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:53.980940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-091225e7-ff34-4719-b8e5-6efefe5a4425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:53.994190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 467/921] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:54.045086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:54.056812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 455/922] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:54.267243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff8e6a-91db-4b01-89d9-3b1746db2f6d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.327775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091225e7-ff34-4719-b8e5-6efefe5a4425 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.367717 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:54.375045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff8e6a-91db-4b01-89d9-3b1746db2f6d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.411835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.412867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.413354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.413849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.414639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:54.416825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:54.426358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:54.427413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:12:54.428747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:54.432258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 468/923] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:54.438269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.444283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091225e7-ff34-4719-b8e5-6efefe5a4425 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.479890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.480521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.480802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.481077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.492030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:54.497329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:54.498331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:12:54.499213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:54.503438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 456/924] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 12:12:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:54.504631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.553043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04f225-3a0c-46da-afa0-e2b8ada16cf5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.557647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:54.590282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.591291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.591797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.592498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.594288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:54.594827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:54.596401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:54.602075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 469/925] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:54.614185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.614381 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb96c4-5fd3-4ec2-8909-1b4415744509 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.650486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.651049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:54.651161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.651394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.651666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.652328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:54.652625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:54.653993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:54.676492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 457/926] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 12:12:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:54.678360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.740665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbef2f4-423d-4bed-bd8a-95eb18ffb28f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.777151 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.777580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.777858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.778123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.778857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:54.779070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:54.780561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:54.784349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1670fea-8d29-49bd-b58a-c8f9e022917f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.789973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 470/927] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 12:12:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:54.795667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.798233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:54.819423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.819834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.820108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.820381 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.831452 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:54.832892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:54.847810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:54.848818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:54.849753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:54.849753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:54.849753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:54.854235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 458/928] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:54.855442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:54.911166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd0c8f-ce5b-4c6c-9d46-417977617b4f None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.923074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:54.951030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b119f61-1536-4c46-b159-342cfc73edc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.952141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b119f61-1536-4c46-b159-342cfc73edc4 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.952628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b119f61-1536-4c46-b159-342cfc73edc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.953128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b119f61-1536-4c46-b159-342cfc73edc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.953969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b119f61-1536-4c46-b159-342cfc73edc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:54.955680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b119f61-1536-4c46-b159-342cfc73edc4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:54.958915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501642-c5bf-40fc-8fce-1098428e87b0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:54.995696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646921a-b65d-427f-a4e6-01563b1be44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:54.996299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646921a-b65d-427f-a4e6-01563b1be44f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:54.996579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646921a-b65d-427f-a4e6-01563b1be44f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:54.996847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646921a-b65d-427f-a4e6-01563b1be44f None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:54.997485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646921a-b65d-427f-a4e6-01563b1be44f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:54.997787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646921a-b65d-427f-a4e6-01563b1be44f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:54.999184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646921a-b65d-427f-a4e6-01563b1be44f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.011311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 459/929] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:55.013635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57a22-4434-43a2-8193-b0d9a0dcc496 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.013917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57a22-4434-43a2-8193-b0d9a0dcc496 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.014190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57a22-4434-43a2-8193-b0d9a0dcc496 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.015609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 460/930] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:55.016977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57a22-4434-43a2-8193-b0d9a0dcc496 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.043673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:55.070430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 471/931] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:12:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:12:55.071951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a613a-86ee-49d7-9e9b-0790042a3446 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.072330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a613a-86ee-49d7-9e9b-0790042a3446 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.072679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a613a-86ee-49d7-9e9b-0790042a3446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.095229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-de2a613a-86ee-49d7-9e9b-0790042a3446 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:55.097457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-de2a613a-86ee-49d7-9e9b-0790042a3446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:55.149014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57a22-4434-43a2-8193-b0d9a0dcc496 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:55.194855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:55.194855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.194855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.194855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.196217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 472/932] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:55.202913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:55.204507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:55.218690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:55.219729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:55.220672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:55.221205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 461/933] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:55.226799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a613a-86ee-49d7-9e9b-0790042a3446 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.305121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:55.330563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a613a-86ee-49d7-9e9b-0790042a3446 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:55.367269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:55.367876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.368156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.368474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.369212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:55.369520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:55.370977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.394897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 473/934] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:55.396243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.491141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.499388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:55.519289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ddb16-c755-4d95-a076-93ef40bc82ee None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:55.555317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38aa42-58f3-4879-b8d4-e05b998338d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:55.556781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38aa42-58f3-4879-b8d4-e05b998338d4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.557059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38aa42-58f3-4879-b8d4-e05b998338d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.557316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38aa42-58f3-4879-b8d4-e05b998338d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.567783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38aa42-58f3-4879-b8d4-e05b998338d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:55.569210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38aa42-58f3-4879-b8d4-e05b998338d4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-0a38aa42-58f3-4879-b8d4-e05b998338d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:55.583419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:55.584451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:55.585364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:55.585364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:55.585364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:55.585364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 462/935] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:55.586130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093566a-7b33-475a-8562-00de4b8e0946 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.586130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093566a-7b33-475a-8562-00de4b8e0946 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.586505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093566a-7b33-475a-8562-00de4b8e0946 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.587897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 463/936] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:55.591339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093566a-7b33-475a-8562-00de4b8e0946 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.606102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015014-b2cd-48aa-a825-b7de6a588b52 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:55.638228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:55.644873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ed05f-dc60-4bf3-9e10-aaa04812edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:55.645558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ed05f-dc60-4bf3-9e10-aaa04812edf3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.645837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ed05f-dc60-4bf3-9e10-aaa04812edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.646108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ed05f-dc60-4bf3-9e10-aaa04812edf3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.646749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ed05f-dc60-4bf3-9e10-aaa04812edf3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:55.648113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ed05f-dc60-4bf3-9e10-aaa04812edf3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-bd5ed05f-dc60-4bf3-9e10-aaa04812edf3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:55.657557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:55.658601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:55.664594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 474/937] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:55.665468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8ab6b-8754-46a5-988c-78732c31f5a5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.665468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8ab6b-8754-46a5-988c-78732c31f5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.665468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8ab6b-8754-46a5-988c-78732c31f5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.683235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-57a8ab6b-8754-46a5-988c-78732c31f5a5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:55.684761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-57a8ab6b-8754-46a5-988c-78732c31f5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:55.709619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093566a-7b33-475a-8562-00de4b8e0946 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:55.739770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:55.745847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:55.750674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.750674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.750674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.750674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:55.750674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:55.750674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.762151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 464/938] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:55.763179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.786677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 475/939] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:55.786677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8ab6b-8754-46a5-988c-78732c31f5a5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.896404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9e241-792b-4169-b155-956a54e97e7e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:55.911997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8ab6b-8754-46a5-988c-78732c31f5a5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:55.945206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:55.948417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.948417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.948417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.948417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:55.948417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:55.948791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.950781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:55.958167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 465/940] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:55.959850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:55.967486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:55.968431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:55.968923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:55.969413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:55.976917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:55.979203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:55.991695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:55.992689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:55.994238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:55.996408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 476/941] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:56.003256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:56.020267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:56.066876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c2be4-dbd4-4755-8baa-a5a506ca223e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.107301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.108171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.108608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.109038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.109836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:56.111128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.112652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.117403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 466/942] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:56.118417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:56.143505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b9815-309f-419f-856d-62970ae6a579 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.186414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.187153 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.187605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.187973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.188787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:56.189213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.190834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.210200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 477/943] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:56.211330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:56.252264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245abc9-e601-4431-a51e-1ad79907c698 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.265468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:56.276050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:56.294771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.295427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.296051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.296331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.296924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.299409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:56.310388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:56.317633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:56.318491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 467/944] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:56.319334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:56.334166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d64fec-bb4c-4d0a-b566-500161e207c4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.375356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.376159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.376442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.376713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/44144b71637548c0b31368fc8114b6b6/roles/37c9983c4e5b4734a384f21bf66503e4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.379155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=44144b71637548c0b31368fc8114b6b6, role_id=37c9983c4e5b4734a384f21bf66503e4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.380642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.441927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e532aa0-b40c-48a6-bf62-4f77025f2226 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.478109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.478109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.478109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.478109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.478544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:56.478544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.478544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.479022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 478/945] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:12:56 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/44144b71637548c0b31368fc8114b6b6/roles/37c9983c4e5b4734a384f21bf66503e4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:12:56.485198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 468/946] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:12:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:56.491067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:56.609183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420c496-fec8-410b-ad08-cabe740b0c17 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.617467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:56.646828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.647439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.647715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.647985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.648615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:56.648914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.650361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.656181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 479/947] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:12:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:56.662465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:56.718418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:56.772712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91789a28-9f1e-4e1c-bd08-257276b4bf9d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.808101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097828e-3a50-4f27-9d51-1eb4123d86d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.808983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097828e-3a50-4f27-9d51-1eb4123d86d3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.809427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097828e-3a50-4f27-9d51-1eb4123d86d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.809843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097828e-3a50-4f27-9d51-1eb4123d86d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.810724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097828e-3a50-4f27-9d51-1eb4123d86d3 None admin] RBAC: Adding query filter params (interface=public, service_id=0748b287bbdb4d02a964d8f5ffa0b753, region_id=RegionOne) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:56.811198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097828e-3a50-4f27-9d51-1eb4123d86d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.812720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097828e-3a50-4f27-9d51-1eb4123d86d3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.822752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 469/948] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:12:56 2022] GET /identity/v3/endpoints?service_id=0748b287bbdb4d02a964d8f5ffa0b753&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:56.824948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:56.930906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e61eb7-103b-417d-bfda-a48ae32c9fa1 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:56.966504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897ba77-4597-4f81-8561-8b051f85b103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:56.967428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ba77-4597-4f81-8561-8b051f85b103 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:56.967889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ba77-4597-4f81-8561-8b051f85b103 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:56.968342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ba77-4597-4f81-8561-8b051f85b103 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:56.969685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897ba77-4597-4f81-8561-8b051f85b103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:56.971280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897ba77-4597-4f81-8561-8b051f85b103 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:56.982774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 480/949] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:12:56 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:12:57.151249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ebb3-71ba-40e6-b1ef-42d4a94eab54 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:57.151744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ebb3-71ba-40e6-b1ef-42d4a94eab54 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:57.151821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ebb3-71ba-40e6-b1ef-42d4a94eab54 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:57.153392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 470/950] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:57.158729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45278908-b31c-4116-afe5-2c8f53165a25 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:57.159017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45278908-b31c-4116-afe5-2c8f53165a25 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:57.159293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45278908-b31c-4116-afe5-2c8f53165a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:57.177431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-45278908-b31c-4116-afe5-2c8f53165a25 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:57.178630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-45278908-b31c-4116-afe5-2c8f53165a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:57.206956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:57.249597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 481/951] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:57.516763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6ebb3-71ba-40e6-b1ef-42d4a94eab54 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:57.622996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6ebb3-71ba-40e6-b1ef-42d4a94eab54 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:57.636473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:57.657269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:57.657866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:57.658140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:57.658406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:57.659076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:57.660556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:57.663162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING keystone.server.flask.common [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 12:12:57.708239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.sql.core [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:12:57.708239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 12:12:57.708239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [parameters: {'id': '53896d254feb4da8a7bffeb411bae405', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 12:12:57.708239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81681) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 53896d254feb4da8a7bffeb411bae405.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 53896d254feb4da8a7bffeb411bae405. Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:12:57.725077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result.read() Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 12:12:57.725900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application yield resource Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise self.value Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application session.commit() Aug 24 12:12:57.726870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:12:57.728348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:12:57.729245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result.read() Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:12:57.730048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [parameters: {'id': '53896d254feb4da8a7bffeb411bae405', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:57.732517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:57.734750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 53896d254feb4da8a7bffeb411bae405. Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:57.736735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 471/952] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Aug 24 12:12:57 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:57.737464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45278908-b31c-4116-afe5-2c8f53165a25 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:57.838981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45278908-b31c-4116-afe5-2c8f53165a25 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:57.875081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b33a70-36c9-43be-a82b-d8eaf7044b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:57.881416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33a70-36c9-43be-a82b-d8eaf7044b7d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:57.881416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33a70-36c9-43be-a82b-d8eaf7044b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:57.881416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b33a70-36c9-43be-a82b-d8eaf7044b7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:57.883269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b33a70-36c9-43be-a82b-d8eaf7044b7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:57.884731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b33a70-36c9-43be-a82b-d8eaf7044b7d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-e0b33a70-36c9-43be-a82b-d8eaf7044b7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:12:57.896969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:57.898961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:57.901741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 482/953] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:57.906675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:58.012475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c556ba-5687-4c3c-b882-2ed399387c59 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:58.046426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b365bad-aaba-4d4d-9d26-56d6abdcbe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:58.047061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b365bad-aaba-4d4d-9d26-56d6abdcbe37 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:58.047339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b365bad-aaba-4d4d-9d26-56d6abdcbe37 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:58.047610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b365bad-aaba-4d4d-9d26-56d6abdcbe37 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:58.048273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b365bad-aaba-4d4d-9d26-56d6abdcbe37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:58.048569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b365bad-aaba-4d4d-9d26-56d6abdcbe37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:58.050312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b365bad-aaba-4d4d-9d26-56d6abdcbe37 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:58.072758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 472/954] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:12:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:58.108720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ef44a-03fd-4908-8e1f-6d8a5f100f9a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:58.108720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ef44a-03fd-4908-8e1f-6d8a5f100f9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:58.108720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ef44a-03fd-4908-8e1f-6d8a5f100f9a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:58.108720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 483/955] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:12:58.115451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104046a-f00b-4a08-a3e3-0674d2c17f58 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:58.115777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104046a-f00b-4a08-a3e3-0674d2c17f58 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:58.115995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104046a-f00b-4a08-a3e3-0674d2c17f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:58.138041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f104046a-f00b-4a08-a3e3-0674d2c17f58 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:58.139666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f104046a-f00b-4a08-a3e3-0674d2c17f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:58.167121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:58.216841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 473/956] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:12:58.491469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ef44a-03fd-4908-8e1f-6d8a5f100f9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:58.518383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:58.549421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576280-ebfd-4c22-8ada-7913d0d02a1a None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:58.549421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576280-ebfd-4c22-8ada-7913d0d02a1a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:58.549421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576280-ebfd-4c22-8ada-7913d0d02a1a None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:58.549421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 474/957] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:12:58.555382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535ad6f-8eb8-40d9-b837-907a078ef7cf None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:58.556568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535ad6f-8eb8-40d9-b837-907a078ef7cf None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:58.557082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535ad6f-8eb8-40d9-b837-907a078ef7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:58.575510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-4535ad6f-8eb8-40d9-b837-907a078ef7cf None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:58.577139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4535ad6f-8eb8-40d9-b837-907a078ef7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:58.601687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ef44a-03fd-4908-8e1f-6d8a5f100f9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:58.642425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bce0e-a4ab-46c9-97af-909b03dd5db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:58.643460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bce0e-a4ab-46c9-97af-909b03dd5db2 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:58.643909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bce0e-a4ab-46c9-97af-909b03dd5db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:58.644331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bce0e-a4ab-46c9-97af-909b03dd5db2 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:58.645140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bce0e-a4ab-46c9-97af-909b03dd5db2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:58.646700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bce0e-a4ab-46c9-97af-909b03dd5db2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:58.663254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 475/958] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:58.715837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 484/959] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:12:58 2022] POST /identity/v3/services => generated 242 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:12:58.931311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535ad6f-8eb8-40d9-b837-907a078ef7cf None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.003126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18390-967b-4493-a1db-6741ec88ff53 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.003681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18390-967b-4493-a1db-6741ec88ff53 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.003681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18390-967b-4493-a1db-6741ec88ff53 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.005276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 485/960] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:12:59.011013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ddf6e-e3aa-477d-a9a4-06a822b923da None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.011288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ddf6e-e3aa-477d-a9a4-06a822b923da None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.011556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ddf6e-e3aa-477d-a9a4-06a822b923da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.021516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:59.032860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-998ddf6e-e3aa-477d-a9a4-06a822b923da None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:12:59.033978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535ad6f-8eb8-40d9-b837-907a078ef7cf None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.034103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-998ddf6e-e3aa-477d-a9a4-06a822b923da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:12:59.089446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.090079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.090354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.090620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.104578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:12:59.109504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:59.110487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:59.111376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 476/961] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 12:12:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:59.117821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.138764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 486/962] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:12:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:12:59.225713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c530a18-0707-43c8-bb70-ca9a8b20edc2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.264480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.265439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.265942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.266407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.267369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:59.267883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:59.269452 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:59.278979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 477/963] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:59.284812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ddf6e-e3aa-477d-a9a4-06a822b923da None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.300248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:59.390026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ddf6e-e3aa-477d-a9a4-06a822b923da None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.412483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.426115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:59.432872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.433811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.434093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.434366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.448268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:59.450390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:12:59.470393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:59.471598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:12:59.472938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:12:59.472938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:12:59.472938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:59.472938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 487/964] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:59.479571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.548304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:59.556192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c59ec8-eb7b-4d3e-8d8f-98447721ace0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.593404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.593792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d957b-768c-4b46-bc69-087e8702725e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.594072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.594351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.594619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.595220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:59.596629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:59.605404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:59.611757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:12:59.612651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 478/965] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:12:59.613460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.643292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.643933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.644215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.644483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.645462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:59.645462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:59.646873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:59.663870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 488/966] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:12:59.664784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.707977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:59.745018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36585c-ce87-4606-b334-415ca2c7180b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.789027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af2e3d-6153-4965-9ace-08672407cdee None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.790805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.792515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.792975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.793425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.794232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:59.794795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:59.796422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:59.808285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 479/967] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:12:59.809924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.823331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:59.832595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.836824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.836824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.836824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.841823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:59.843414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:12:59.856875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:12:59.857929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:12:59.859007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:12:59.862195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 489/968] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:12:59.867427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:12:59.933795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36f9a-ec34-41bf-9517-f3cd13b60bcc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.977955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:12:59.978958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:12:59.979397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:12:59.980045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:12:59.981263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:12:59.982222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:12:59.984412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:12:59.986139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:12:59.990498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072c5b2-4f15-4c97-bbe3-944c81f27d7a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:12:59.991955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 480/969] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:00.003155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:00.027769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:00.028508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.028864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.029208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.030041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:00.030416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:00.031989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:00.054735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 490/970] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 12:12:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:00.057596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:00.107720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:00.128360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e5cec-59db-4126-934d-3a17927f17ab None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:00.165215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f23852-0921-46ca-8d18-1db89506e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:00.165974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f23852-0921-46ca-8d18-1db89506e311 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.166248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f23852-0921-46ca-8d18-1db89506e311 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.166853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f23852-0921-46ca-8d18-1db89506e311 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.167657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f23852-0921-46ca-8d18-1db89506e311 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:00.169089 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f23852-0921-46ca-8d18-1db89506e311 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:00.169187 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001607b9-97c7-40c0-9013-90146c208fe8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:00.203697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:00.204350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.204627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.204898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.246115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 481/971] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:13:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:00.247506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5771ae-35b9-43b9-a783-705ebdcfa4e5 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.247800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5771ae-35b9-43b9-a783-705ebdcfa4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.248208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5771ae-35b9-43b9-a783-705ebdcfa4e5 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.249612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 482/972] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:00.255617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010987b-45bf-426b-bb00-ec245a9e799c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.255953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010987b-45bf-426b-bb00-ec245a9e799c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.256231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010987b-45bf-426b-bb00-ec245a9e799c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.280227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2010987b-45bf-426b-bb00-ec245a9e799c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:00.282192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2010987b-45bf-426b-bb00-ec245a9e799c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:00.287727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:00.289611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:00.342151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:00.344740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:00.375180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 483/973] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:00.407268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 491/974] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:00 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:00.412835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010987b-45bf-426b-bb00-ec245a9e799c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:00.521992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010987b-45bf-426b-bb00-ec245a9e799c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:00.557570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:00.558503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.558946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.559356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.570124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:00.571830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:00.586771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:00.587963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:00.589069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:00.589069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:00.589069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:00.590732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 484/975] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:00.596572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:00.648555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:00.685323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:00.708016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5231ce-05d2-4990-8574-80b9a273e3f6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:00.748458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc605-9d79-4d14-8d32-e0aae86049e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:00.749132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc605-9d79-4d14-8d32-e0aae86049e8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.749492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc605-9d79-4d14-8d32-e0aae86049e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.749767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc605-9d79-4d14-8d32-e0aae86049e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.750454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc605-9d79-4d14-8d32-e0aae86049e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:00.751248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc605-9d79-4d14-8d32-e0aae86049e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:00.752955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc605-9d79-4d14-8d32-e0aae86049e8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:00.772932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 492/976] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:00.773135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea3cc-e9f4-4970-949a-e1769ed6946b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:00.773220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d581eb3-f409-4393-ac70-bf65b9bae1d8 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.773220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d581eb3-f409-4393-ac70-bf65b9bae1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.773220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d581eb3-f409-4393-ac70-bf65b9bae1d8 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.773659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 493/977] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:00.774250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d581eb3-f409-4393-ac70-bf65b9bae1d8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:00.778576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:00.809411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:00.810016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.810297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.810568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.811250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:00.812688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:00.856358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.sql.core [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:13:00.856358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 12:13:00.856358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [parameters: {'user_id': '7a483b94409b4894b7b334a1a2499272', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 12:13:00.856358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81681) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-14e9a9d1-07da-4dc1-bee3-973c3fceecfe None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result.read() Aug 24 12:13:00.863751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:13:00.871130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application yield resource Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise self.value Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application session.commit() Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:13:00.872114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:13:00.873638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:13:00.874559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application result.read() Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:13:00.875559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [parameters: {'user_id': '7a483b94409b4894b7b334a1a2499272', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:00.876428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 12:13:00.877295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 485/978] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 12:13:00 2022] POST /identity/v3/users => generated 160 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fb7a7-942f-4906-bbbd-fd750b817d9b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fb7a7-942f-4906-bbbd-fd750b817d9b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.880082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fb7a7-942f-4906-bbbd-fd750b817d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.888161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d581eb3-f409-4393-ac70-bf65b9bae1d8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:00.893355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-bb4fb7a7-942f-4906-bbbd-fd750b817d9b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:00.894907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-bb4fb7a7-942f-4906-bbbd-fd750b817d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:00.924801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:00.924801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:00.924801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:00.925133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:00.932534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:00.933966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:13:00.945200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:00.946161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 494/979] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:13:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:00.947856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:00.972072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 486/980] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:00.972936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fb7a7-942f-4906-bbbd-fd750b817d9b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:01.012806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:01.015120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:01.063208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8380db-5278-481c-99de-2657f1df157a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:01.108912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:01.110054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:01.110545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:01.111132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:01.111387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fb7a7-942f-4906-bbbd-fd750b817d9b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:01.130701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:01.133043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:13:01.146977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:01.147976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:13:01.148939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:01.157807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:01.158483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 495/981] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:13:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:01.158765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:01.159044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:01.159366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:01.160076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:01.160404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:01.160790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:01.162044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:01.186814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 487/982] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:13:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:01.189001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:01.279008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edbe4e-17b5-4287-91be-d5187931993b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:01.301937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:01.303681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:01.315350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:01.315997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:01.316277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:01.316582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:01.317241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:01.317541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:01.318674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fadce-f1d8-49a0-bf95-e624fe0c7ef9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:01.318995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:01.353949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 496/983] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 12:13:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:01.355792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:01.374527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:01.375419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:01.375774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:01.376120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:01.377011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] RBAC: Adding query filter params (role.id=37c9983c4e5b4734a384f21bf66503e4, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=44144b71637548c0b31368fc8114b6b6) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:01.377418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:01.379770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:01.396350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 488/984] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:01 2022] GET /identity/v3/role_assignments?user.id=44144b71637548c0b31368fc8114b6b6&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=37c9983c4e5b4734a384f21bf66503e4 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:01.504198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8a579-3f00-4ccc-910f-04fef833df38 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:01.542575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:01.543252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:01.543585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:01.543863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] PATH_INFO: `/v3/services/compute` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:01.544460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:01.545900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:01.555322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:01.556976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:01.557818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 497/985] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:13:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:01.562986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:01.678776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659ef4a-3bb4-4c33-8b74-e36a4e275ed9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:01.711217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:01.712375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:01.712650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:01.712915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:01.713535 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:01.713829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:01.715209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:01.720180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 489/986] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:13:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:01.726023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:01.776217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:01.829937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7bf9-e61c-4e66-b010-e91c7fde31ba None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:01.865062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:01.865672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:01.865949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:01.866220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:01.866874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:01.867177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:01.868626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:01.878690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 498/987] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:13:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:01.879947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:01.895274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:01.984304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f537e-6b81-40fe-917b-d70ce162c1a0 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:02.021240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e799d-4076-4a00-8738-5557b15f5147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:02.022179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e799d-4076-4a00-8738-5557b15f5147 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:02.022723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e799d-4076-4a00-8738-5557b15f5147 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:02.023234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e799d-4076-4a00-8738-5557b15f5147 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:02.024183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e799d-4076-4a00-8738-5557b15f5147 None admin] RBAC: Adding query filter params (interface=public, service_id=143ebd3769634497b83079be247c6f38, region_id=RegionOne) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:02.025261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e799d-4076-4a00-8738-5557b15f5147 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:02.026891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e799d-4076-4a00-8738-5557b15f5147 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:02.032558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 490/988] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:13:01 2022] GET /identity/v3/endpoints?service_id=143ebd3769634497b83079be247c6f38&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:02.037749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:02.138743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bcd6e-ea91-42d2-a4f0-6202dcf070cb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:02.149047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:02.172494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1db80-91ed-40fd-9d2d-481c651149c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:02.173088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1db80-91ed-40fd-9d2d-481c651149c5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:02.173360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1db80-91ed-40fd-9d2d-481c651149c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:02.173625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1db80-91ed-40fd-9d2d-481c651149c5 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:02.174212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1db80-91ed-40fd-9d2d-481c651149c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:02.175584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1db80-91ed-40fd-9d2d-481c651149c5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:02.182479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 499/989] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:13:02 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:13:02.324263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853276cd-7d22-4644-8435-db324e70dbed None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:02.324770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853276cd-7d22-4644-8435-db324e70dbed None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:02.324770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853276cd-7d22-4644-8435-db324e70dbed None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:02.326212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 491/990] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:02.331515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730e62e-78b3-4e8e-b2b8-f5d1386a1ad5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:02.331798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730e62e-78b3-4e8e-b2b8-f5d1386a1ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:02.332074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730e62e-78b3-4e8e-b2b8-f5d1386a1ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:02.350394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3730e62e-78b3-4e8e-b2b8-f5d1386a1ad5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:02.351678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3730e62e-78b3-4e8e-b2b8-f5d1386a1ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:02.431030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 500/991] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:02.694796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853276cd-7d22-4644-8435-db324e70dbed None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:02.773390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:02.804606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853276cd-7d22-4644-8435-db324e70dbed None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:02.838697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:02.839327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:02.839605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:02.839876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:02.840447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:02.841805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:02.850344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:02.851508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:02.852918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 492/992] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:13:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:02.860155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730e62e-78b3-4e8e-b2b8-f5d1386a1ad5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:02.960402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730e62e-78b3-4e8e-b2b8-f5d1386a1ad5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:02.962827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:02.994898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:02.995492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:02.995766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:02.996084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:02.996704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:02.996998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:02.998348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:03.006805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 501/993] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:13:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:03.009949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:03.111879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0af9c-5eda-40ab-97fd-98cd2136ac6b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:03.144168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:03.144775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:03.145053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:03.145322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:03.145950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:03.146248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:03.147640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:03.157935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 493/994] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:13:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:03.159106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:03.258456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f49ee-8131-4ffa-8382-6002e4d4b779 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:03.294205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d91a1a-1f05-47dc-9c7c-deb9bf6106fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:03.294829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d91a1a-1f05-47dc-9c7c-deb9bf6106fe None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:03.295107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d91a1a-1f05-47dc-9c7c-deb9bf6106fe None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:03.295376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d91a1a-1f05-47dc-9c7c-deb9bf6106fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:03.296082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d91a1a-1f05-47dc-9c7c-deb9bf6106fe None admin] RBAC: Adding query filter params (interface=admin, service_id=0748b287bbdb4d02a964d8f5ffa0b753, region_id=RegionOne) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:03.296387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d91a1a-1f05-47dc-9c7c-deb9bf6106fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:03.297742 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d91a1a-1f05-47dc-9c7c-deb9bf6106fe None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:03.307663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 502/995] 10.222.0.45 () {56 vars in 1249 bytes} [Wed Aug 24 12:13:03 2022] GET /identity/v3/endpoints?service_id=0748b287bbdb4d02a964d8f5ffa0b753&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:03.308631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:03.312419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:03.393230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbc11b-9696-49b4-98ed-6e037a0065a3 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:03.393598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbc11b-9696-49b4-98ed-6e037a0065a3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:03.393768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbc11b-9696-49b4-98ed-6e037a0065a3 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:03.395260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 503/996] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:13:03.400579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ccb4e-6b69-4bb7-af2e-9ed7c79174b6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:03.400943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ccb4e-6b69-4bb7-af2e-9ed7c79174b6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:03.401282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ccb4e-6b69-4bb7-af2e-9ed7c79174b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:03.411160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9823323-7184-4160-830d-d2040c8c6fa2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:03.419812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f84ccb4e-6b69-4bb7-af2e-9ed7c79174b6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:03.421061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f84ccb4e-6b69-4bb7-af2e-9ed7c79174b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:03.448522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:03.454176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:03.454883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:03.455277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:03.455748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:03.457235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:03.458044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:03.465763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 494/997] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:13:03 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:13:03.503226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 504/998] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:03.792468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:03.862548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:03.897073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd096a7-0600-479e-b91a-94161002d627 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:03.932079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ef5d4-b3f9-4c0f-93d9-cf0b44e2ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:03.932705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ef5d4-b3f9-4c0f-93d9-cf0b44e2ea92 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:03.932978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ef5d4-b3f9-4c0f-93d9-cf0b44e2ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:03.933244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ef5d4-b3f9-4c0f-93d9-cf0b44e2ea92 None admin] PATH_INFO: `/v3/services/image` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:03.933823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ef5d4-b3f9-4c0f-93d9-cf0b44e2ea92 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:03.935210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ef5d4-b3f9-4c0f-93d9-cf0b44e2ea92 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-941ef5d4-b3f9-4c0f-93d9-cf0b44e2ea92 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:03.943872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:03.944871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:03.945782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:03.950421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 495/999] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 24 12:13:03 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:03.951912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ccb4e-6b69-4bb7-af2e-9ed7c79174b6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:04.051523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ccb4e-6b69-4bb7-af2e-9ed7c79174b6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:04.095296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:04.095902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:04.096178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:04.096446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:04.097088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] RBAC: Adding query filter params (name=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:04.097389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:04.098906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:04.104299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 505/1000] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:04.105027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa9e8b-f2fd-4db9-8307-4cdd1aa552a8 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:04.105318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa9e8b-f2fd-4db9-8307-4cdd1aa552a8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:04.105598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa9e8b-f2fd-4db9-8307-4cdd1aa552a8 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:04.107362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 496/1001] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:04.110042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:04.112037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44c375-f1b5-4393-ac2c-89474941ac79 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:04.112321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44c375-f1b5-4393-ac2c-89474941ac79 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:04.112596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44c375-f1b5-4393-ac2c-89474941ac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:04.130770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1a44c375-f1b5-4393-ac2c-89474941ac79 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:04.131964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1a44c375-f1b5-4393-ac2c-89474941ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:04.141418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:04.198784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 497/1002] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:04.216585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def40a8b-bbc4-4040-9ab8-e0a105c0802e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:04.250210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:04.250823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:04.251096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:04.251361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:04.251975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] RBAC: Adding query filter params (type=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:04.252270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:04.253601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:04.258012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 506/1003] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:04.468285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44c375-f1b5-4393-ac2c-89474941ac79 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:04.532140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:04.573720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44c375-f1b5-4393-ac2c-89474941ac79 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:04.608659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:04.609288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:04.609571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:04.609844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:04.610424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:04.611816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:04.620606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:04.621640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:04.622515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 498/1004] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:13:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:04.631668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:04.722255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:04.732948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e2a2f-be97-4f1c-bc41-d25aa447d04b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:04.766769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:04.767371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:04.767652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:04.767927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:04.768553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:04.768856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:04.770218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:04.777795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 507/1005] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:13:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:04.781392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:04.889811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfc3c9-4fb8-41ac-abd2-d2fc087321cc None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:04.916246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:04.925295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf22995-f886-4b12-b6b8-a40e2f18d928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:04.925897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf22995-f886-4b12-b6b8-a40e2f18d928 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:04.926178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf22995-f886-4b12-b6b8-a40e2f18d928 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:04.926448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf22995-f886-4b12-b6b8-a40e2f18d928 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:04.927113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf22995-f886-4b12-b6b8-a40e2f18d928 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:04.927414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf22995-f886-4b12-b6b8-a40e2f18d928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:04.928773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf22995-f886-4b12-b6b8-a40e2f18d928 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:04.933923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 499/1006] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 12:13:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:04.939610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:05.042910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00344d6-3794-4d4a-9ca6-ae1a8fba3a23 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:05.077065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c4667-e6d6-4251-aac9-322f3cd7a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:05.077671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c4667-e6d6-4251-aac9-322f3cd7a637 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:05.077952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c4667-e6d6-4251-aac9-322f3cd7a637 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:05.078227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c4667-e6d6-4251-aac9-322f3cd7a637 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:05.078924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c4667-e6d6-4251-aac9-322f3cd7a637 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:05.080293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c4667-e6d6-4251-aac9-322f3cd7a637 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:05.166104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 508/1007] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:13:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:13:05.389293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76058623-f89d-4237-adae-d345da3c22c4 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:05.389640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76058623-f89d-4237-adae-d345da3c22c4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:05.389820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76058623-f89d-4237-adae-d345da3c22c4 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:05.391231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 500/1008] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:05.396650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316e858-8209-4e86-8951-168d6fb5ed06 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:05.397083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316e858-8209-4e86-8951-168d6fb5ed06 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:05.397250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316e858-8209-4e86-8951-168d6fb5ed06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:05.416934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d316e858-8209-4e86-8951-168d6fb5ed06 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:05.418157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d316e858-8209-4e86-8951-168d6fb5ed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:05.421313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:05.496607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 509/1009] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:05 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:05.769534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76058623-f89d-4237-adae-d345da3c22c4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:05.871643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76058623-f89d-4237-adae-d345da3c22c4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:05.906198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:05.906855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:05.907136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:05.907410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:05.907993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:05.909358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:05.918125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:05.919511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:13:05.920372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:05.922812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 501/1010] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 12:13:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:05.925259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316e858-8209-4e86-8951-168d6fb5ed06 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:06.024887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316e858-8209-4e86-8951-168d6fb5ed06 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:06.059201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c33c90-41ac-4c7b-a87c-4363d148a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:06.059805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c33c90-41ac-4c7b-a87c-4363d148a10b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:06.060083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c33c90-41ac-4c7b-a87c-4363d148a10b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:06.060352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c33c90-41ac-4c7b-a87c-4363d148a10b None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:06.060984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c33c90-41ac-4c7b-a87c-4363d148a10b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:06.061285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c33c90-41ac-4c7b-a87c-4363d148a10b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:06.062639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c33c90-41ac-4c7b-a87c-4363d148a10b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:06.064587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:06.070816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 510/1011] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:13:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:06.074133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:06.133182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:06.134181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99199d34-1bcb-444d-8464-1c1c813e695a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:06.134461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99199d34-1bcb-444d-8464-1c1c813e695a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:06.134763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99199d34-1bcb-444d-8464-1c1c813e695a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:06.136366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 511/1012] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:13:06.141227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07a79-8ad4-4c8e-acbe-b18b890754b4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:06.141502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07a79-8ad4-4c8e-acbe-b18b890754b4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:06.141771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07a79-8ad4-4c8e-acbe-b18b890754b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:06.161571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6eb07a79-8ad4-4c8e-acbe-b18b890754b4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:06.163169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6eb07a79-8ad4-4c8e-acbe-b18b890754b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:06.179592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65638f3d-5325-4f8f-b297-bf8234e318a2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:06.213858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:06.214457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:06.214755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:06.215023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:06.215653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:06.215947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:06.217298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:06.222079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 502/1013] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 12:13:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:06.228274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:06.238080 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 512/1014] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:06 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:06.324699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca69df-502e-4498-b8c9-c162ca8908ce None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:06.341958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:06.360948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b7347-2b34-476d-9d03-a9d3e6a25e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:06.362400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b7347-2b34-476d-9d03-a9d3e6a25e9b None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:06.362400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b7347-2b34-476d-9d03-a9d3e6a25e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:06.362400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b7347-2b34-476d-9d03-a9d3e6a25e9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:06.362935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b7347-2b34-476d-9d03-a9d3e6a25e9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:06.364297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b7347-2b34-476d-9d03-a9d3e6a25e9b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:06.480946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 503/1015] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:13:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:06.503857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07a79-8ad4-4c8e-acbe-b18b890754b4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:06.614922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07a79-8ad4-4c8e-acbe-b18b890754b4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:06.645985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:06.658734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae06a2-670b-4f46-98c1-fdf70bc53ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:06.659333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae06a2-670b-4f46-98c1-fdf70bc53ed0 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:06.659602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae06a2-670b-4f46-98c1-fdf70bc53ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:06.659867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae06a2-670b-4f46-98c1-fdf70bc53ed0 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:06.660507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae06a2-670b-4f46-98c1-fdf70bc53ed0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:06.661910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae06a2-670b-4f46-98c1-fdf70bc53ed0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:06.782451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 513/1016] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:13:06 2022] POST /identity/v3/services => generated 242 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:07.185741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8803840-07e4-4dc0-973e-96de631e4015 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:07.186263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8803840-07e4-4dc0-973e-96de631e4015 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:07.186263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8803840-07e4-4dc0-973e-96de631e4015 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:07.187795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 504/1017] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:07.192765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cc90c-0e9c-4977-bf19-b7de9cb35153 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:07.193052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cc90c-0e9c-4977-bf19-b7de9cb35153 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:07.193332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cc90c-0e9c-4977-bf19-b7de9cb35153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:07.210947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-356cc90c-0e9c-4977-bf19-b7de9cb35153 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:07.212138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-356cc90c-0e9c-4977-bf19-b7de9cb35153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:07.288780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 514/1018] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:07.562480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8803840-07e4-4dc0-973e-96de631e4015 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:07.666564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8803840-07e4-4dc0-973e-96de631e4015 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:07.699945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:07.700578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:07.700858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:07.701132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:07.711674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:07.716401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:07.717321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:13:07.718168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:07.722427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 505/1019] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 12:13:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:07.723710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cc90c-0e9c-4977-bf19-b7de9cb35153 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:07.823665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cc90c-0e9c-4977-bf19-b7de9cb35153 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:07.836754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:07.857845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:07.858442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:07.858747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:07.859017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:07.859671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:07.860059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:07.861420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:07.874716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 515/1020] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 12:13:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:07.876944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:07.919223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:07.986762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf941fc-632e-49c9-bce6-4534076dc85b None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:08.019973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:08.020594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.020868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.021136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.031529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:08.032953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:08.047963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:08.048929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:08.049844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:08.049844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:08.049844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:08.049844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 506/1021] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:08.056738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:08.156044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367a506-fe52-4c72-b508-fbc6db7cb963 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:08.189712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:08.190311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.190591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.190885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.191513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:08.191806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:08.193145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:08.204786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 516/1022] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:08.210222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:08.305882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:08.315902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd559dc8-ae8e-455a-ac47-956215991c92 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:08.350244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd733c-0396-4312-8b54-41338c30912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:08.351202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd733c-0396-4312-8b54-41338c30912b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.351695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd733c-0396-4312-8b54-41338c30912b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.352802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd733c-0396-4312-8b54-41338c30912b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.361085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd733c-0396-4312-8b54-41338c30912b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:08.362741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd733c-0396-4312-8b54-41338c30912b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-a7bd733c-0396-4312-8b54-41338c30912b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:13:08.373930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:08.374942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:13:08.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:08.378868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 507/1023] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:13:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:08.381666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:08.405843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c0035-a565-4617-bf23-125d46fbcf5b None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.406219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c0035-a565-4617-bf23-125d46fbcf5b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.406392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c0035-a565-4617-bf23-125d46fbcf5b None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.407846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 508/1024] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:08.415614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b0da4-ff2c-4f09-ab20-b6a27478fb23 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.415614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b0da4-ff2c-4f09-ab20-b6a27478fb23 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.415614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b0da4-ff2c-4f09-ab20-b6a27478fb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.426776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:08.432773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-908b0da4-ff2c-4f09-ab20-b6a27478fb23 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:08.433963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-908b0da4-ff2c-4f09-ab20-b6a27478fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:08.496716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95557dec-0baa-44fc-864c-c29d60fbdd04 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:08.506548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 509/1025] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:08.536025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a11132-449a-4cc0-8be5-07b81edb96c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:08.536618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a11132-449a-4cc0-8be5-07b81edb96c9 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.536890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a11132-449a-4cc0-8be5-07b81edb96c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.537155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a11132-449a-4cc0-8be5-07b81edb96c9 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.537864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a11132-449a-4cc0-8be5-07b81edb96c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:08.538163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a11132-449a-4cc0-8be5-07b81edb96c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:08.539650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a11132-449a-4cc0-8be5-07b81edb96c9 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:08.562329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 517/1026] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:13:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:08.563177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b0da4-ff2c-4f09-ab20-b6a27478fb23 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:08.650358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:08.672284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b0da4-ff2c-4f09-ab20-b6a27478fb23 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:08.703411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:08.703736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247953-7975-4023-a237-b4f720e6192e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.703931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247953-7975-4023-a237-b4f720e6192e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.704029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.704183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247953-7975-4023-a237-b4f720e6192e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.704293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.704534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.705627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 518/1027] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:13:08.710304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71dd4f-43ed-4f41-801c-2c69b319f47f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.710587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71dd4f-43ed-4f41-801c-2c69b319f47f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.746601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71dd4f-43ed-4f41-801c-2c69b319f47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.746601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0d71dd4f-43ed-4f41-801c-2c69b319f47f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:08.746601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0d71dd4f-43ed-4f41-801c-2c69b319f47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:08.787931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:08.789419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:08.810931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 519/1028] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:08.811680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71dd4f-43ed-4f41-801c-2c69b319f47f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:08.834878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:08.879201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:08.901328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 510/1029] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:13:08.907043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:08.920497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71dd4f-43ed-4f41-801c-2c69b319f47f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:08.958797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:08.959352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:08.959639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:08.959911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:08.961174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:08.963135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:08.965103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 520/1030] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:08.970612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.012467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4571531-ee89-4df6-bc54-2b985452a630 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.048170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.048787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.049061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.049328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.060348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.061774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.067711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:09.077349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:09.078352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:09.088179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:09.088179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:09.088179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:09.088371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb211a5-62f3-4242-b39a-40247d0394f6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.088449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 511/1031] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:09.088449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.112229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.112835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.113109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.113500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.113618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.114017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.115468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.181140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157040a3-d4cf-4e24-8fa5-082cf4614433 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.213293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.213855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.214102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.214366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] PATH_INFO: `/v3/services/image` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.214963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.216303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:09.224596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:09.225883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:09.226805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 512/1032] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:09.227917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.240290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 521/1033] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:13:08 2022] POST /identity/v3/projects => generated 311 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:13:09.241285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.334789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.341007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9245d-8a78-4dd4-bcdf-664b4cd717ab None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.357332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb9522-5873-45c9-b1b8-664a75bd6c47 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.379337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.380406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.380753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.380947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.381587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:09.381880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.383290 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.396325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 513/1034] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:09.399440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.400053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.400333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.400607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.401374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] RBAC: Adding query filter params (name=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:09.402374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.402494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.403088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.409891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 522/1035] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:09.416518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.426183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.517259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba117d7f-b163-46cc-8c8f-8eeb8bef9aa7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.522422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc32cd-d3e6-4c46-bda2-00a409afc892 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.556841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.558000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.558593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.559108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.559232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.559777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.560052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.560321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.560938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] RBAC: Adding query filter params (type=image) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:09.561234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.562612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.567790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 523/1036] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:09.568001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.569234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.573358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:13:09.580263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:09.581187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:09.587005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 514/1037] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:09.587544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.605628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.654627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.686639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afcca2-1bc2-494b-bee5-e30eeef44c6b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.696583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df805a5-064d-46a1-ad68-7642ea0e4bb4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.723047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.723649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.724015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.724265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.724920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] RBAC: Adding query filter params (interface=public, service_id=867aba87f67b4795b3705aa2cbe7af89, region_id=RegionOne) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:09.725318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.726621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.732695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.732851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 524/1038] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/endpoints?service_id=867aba87f67b4795b3705aa2cbe7af89&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:09.733073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.733306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.733582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.734276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:09.734582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.736051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.738000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.749837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 515/1039] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:09.755711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:09.836128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.840845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26531a-d064-4f7f-8f9e-357e772e9ec7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.862762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcc15a-c159-49b2-a9a1-3fcaa8052f5c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:09.875983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5ceab-09ed-44c4-91d7-f21d307bc481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.879421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5ceab-09ed-44c4-91d7-f21d307bc481 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.879421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5ceab-09ed-44c4-91d7-f21d307bc481 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.879421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5ceab-09ed-44c4-91d7-f21d307bc481 None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.879421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5ceab-09ed-44c4-91d7-f21d307bc481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.879421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5ceab-09ed-44c4-91d7-f21d307bc481 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.880248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:09.886422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 525/1040] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:13:09.902267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d048d-96d2-4811-98c3-0b675020c55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:09.907796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d048d-96d2-4811-98c3-0b675020c55c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:09.907796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d048d-96d2-4811-98c3-0b675020c55c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:09.907796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d048d-96d2-4811-98c3-0b675020c55c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:09.907796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d048d-96d2-4811-98c3-0b675020c55c None admin] RBAC: Adding query filter params (role.id=37c9983c4e5b4734a384f21bf66503e4, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=0ed16fea638a45938eee8cdf234333f1) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:09.907796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d048d-96d2-4811-98c3-0b675020c55c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:09.907796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d048d-96d2-4811-98c3-0b675020c55c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:09.918713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 516/1041] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:09 2022] GET /identity/v3/role_assignments?user.id=0ed16fea638a45938eee8cdf234333f1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=37c9983c4e5b4734a384f21bf66503e4 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:11.181914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3069f-5bd4-4a4a-9f11-c5bc995fa98d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.182434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3069f-5bd4-4a4a-9f11-c5bc995fa98d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.182434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3069f-5bd4-4a4a-9f11-c5bc995fa98d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.183857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 526/1042] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:11.189225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e46c8-3b56-4bbe-a5d5-63b07daca9bd None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.189691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e46c8-3b56-4bbe-a5d5-63b07daca9bd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.189772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e46c8-3b56-4bbe-a5d5-63b07daca9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.208329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cc1e46c8-3b56-4bbe-a5d5-63b07daca9bd None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:11.209527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cc1e46c8-3b56-4bbe-a5d5-63b07daca9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:11.279095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 517/1043] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:11.544344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3069f-5bd4-4a4a-9f11-c5bc995fa98d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:11.649679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3069f-5bd4-4a4a-9f11-c5bc995fa98d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:11.683539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d1cf4-5742-4f89-ae39-61ee81292265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:11.684156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d1cf4-5742-4f89-ae39-61ee81292265 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.684433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d1cf4-5742-4f89-ae39-61ee81292265 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.684701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d1cf4-5742-4f89-ae39-61ee81292265 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.685810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d1cf4-5742-4f89-ae39-61ee81292265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:11.687272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d1cf4-5742-4f89-ae39-61ee81292265 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:11.688991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 527/1044] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:11.694555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e46c8-3b56-4bbe-a5d5-63b07daca9bd None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:11.764832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:11.794937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e46c8-3b56-4bbe-a5d5-63b07daca9bd None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:11.813199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f023326-dd26-4053-9de3-9b7a59b05f1a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.813480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f023326-dd26-4053-9de3-9b7a59b05f1a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.813760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f023326-dd26-4053-9de3-9b7a59b05f1a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.815268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 528/1045] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:11.820263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31340d-6447-44a9-b14b-fecb75fd8b3c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.820551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31340d-6447-44a9-b14b-fecb75fd8b3c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.821400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31340d-6447-44a9-b14b-fecb75fd8b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.832391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:11.832998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.833276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.833545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.834203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:11.835659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:11.840031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2f31340d-6447-44a9-b14b-fecb75fd8b3c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:11.841212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2f31340d-6447-44a9-b14b-fecb75fd8b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:11.906371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:11.916113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 529/1046] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:11.917348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177901-1a6d-4b96-bf5c-f76d95909c5f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.917646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177901-1a6d-4b96-bf5c-f76d95909c5f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.917902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177901-1a6d-4b96-bf5c-f76d95909c5f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.919188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 530/1047] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:11.925688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c5ed-4eb5-4222-b69b-58f2e173f673 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:11.926169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c5ed-4eb5-4222-b69b-58f2e173f673 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:11.926641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c5ed-4eb5-4222-b69b-58f2e173f673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:11.946388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4098c5ed-4eb5-4222-b69b-58f2e173f673 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:11.947616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4098c5ed-4eb5-4222-b69b-58f2e173f673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:11.948881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 518/1048] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:13:11 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:12.038762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 531/1049] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:12.189704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.294408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94210dc0-dda7-4b19-951c-ba5f7dd3298c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.298845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098c5ed-4eb5-4222-b69b-58f2e173f673 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.320548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:12.331332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.331962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.332246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.332518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] PATH_INFO: `/v3/services/image` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.333097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:12.334462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:12.343447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:12.344468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:12.345320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 519/1050] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:12.350310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.408874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098c5ed-4eb5-4222-b69b-58f2e173f673 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.444851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:12.454622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.455277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.455859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.455859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.459932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a254-8e21-403e-a90e-dff22a044a33 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.466638 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:12.472043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:12.473076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:13:12.473903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:12.476799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 532/1051] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:12.483293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.498896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.499792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.500075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.500402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.501034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] RBAC: Adding query filter params (name=image) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:12.501360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:12.502747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:12.508403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 520/1052] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:12.513943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.572590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:12.596514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19809a-5c96-4842-bfe2-6eb926f4c7ba None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.621081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b372-bac4-49de-adce-295a48142249 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.633774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.634393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.634643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.634955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.635593 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:12.635886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:12.637251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:12.650306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 533/1053] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:12.651042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.655498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.656099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.656372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.656638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.657260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] RBAC: Adding query filter params (type=image) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:12.657553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:12.658932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:12.665422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 521/1054] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:12.670868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.701412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:12.752352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fef77-caa9-4401-9547-4c86409ce9c8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.774821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e585b-381a-451c-a39a-d87117139346 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.786614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.787262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.787535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.787802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.794729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:12.799457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:12.801092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:12.812066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.812665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.812944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.813213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.813866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:12.815254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:12.815843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:12.816987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:12.817883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:12.817883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:12.817883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:12.817883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 534/1055] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:12.822886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:12.891753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 522/1056] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:13:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:13:12.925996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:12.930346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36523e0-a2a9-4ccd-8617-771c1d93cab3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:12.976414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:12.977651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:12.978028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:12.978420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:12.979207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:12.979608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:12.981135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:12.994216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 535/1057] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:12.999861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:13.110293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74894e-a0cd-4ea3-b70d-fbe4c7f6be5e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:13.145626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:13.146253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.146532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.146834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.154486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:13.155961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:13:13.167531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:13.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:13.169428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 523/1058] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:13:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:13.174672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:13.275749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111136a9-17d1-40bc-bdc9-994811b70bfa None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:13.309631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:13.310224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.310494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.310796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.311517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:13.311821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:13.313253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:13.331454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 536/1059] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:13:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:13.336927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:13.372950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:13.445364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6bad0-6281-4270-9221-5539b0e5293a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:13.479803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:13.480434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.480711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.480981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.491963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:13.493937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:13.507932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:13.508948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:13.515165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:13.515165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:13.515165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:13.515165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 524/1060] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:13.515418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:13.531479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:13.615936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78515944-9383-47a0-bf5c-e5dd756e274a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:13.650478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:13.651104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.651381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.651648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.652286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:13.652583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:13.653922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:13.665632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 537/1061] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:13.671074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:13.755578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:13.770637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cfe6a-27ea-46d4-8c7d-a79ec0c96850 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:13.805387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b38f50-1585-45dd-8cde-1a05a964a20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:13.805939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38f50-1585-45dd-8cde-1a05a964a20e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.806218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38f50-1585-45dd-8cde-1a05a964a20e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.806489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b38f50-1585-45dd-8cde-1a05a964a20e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.813057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b38f50-1585-45dd-8cde-1a05a964a20e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:13.814491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b38f50-1585-45dd-8cde-1a05a964a20e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-15b38f50-1585-45dd-8cde-1a05a964a20e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:13:13.825323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:13.826219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:13.827201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:13.830977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 525/1062] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:13.832199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:13.909366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:13.933646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf917-5088-4a3c-8417-89e42b939dd2 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.933931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf917-5088-4a3c-8417-89e42b939dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.934204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cf917-5088-4a3c-8417-89e42b939dd2 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.935594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 526/1063] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:13:13.937990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8b690-e051-4e0a-8825-320c5efb7b66 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:13.941122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc630d-5c8f-4c75-89e6-a4309d4d62af None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.941412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc630d-5c8f-4c75-89e6-a4309d4d62af None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.941692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc630d-5c8f-4c75-89e6-a4309d4d62af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.960719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-41bc630d-5c8f-4c75-89e6-a4309d4d62af None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:13.962591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-41bc630d-5c8f-4c75-89e6-a4309d4d62af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:13.973919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:13.974530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:13.974838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:13.975114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:13.975797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:13.976099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:13.977539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:13.990695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 538/1064] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:13.996555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:14.040881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 527/1065] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:14.100058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e5393-7f54-4c43-9861-752c21ab1290 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:14.133532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:14.134182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:14.134460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:14.134756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/0ed16fea638a45938eee8cdf234333f1/roles/37c9983c4e5b4734a384f21bf66503e4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:14.136877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=0ed16fea638a45938eee8cdf234333f1, role_id=37c9983c4e5b4734a384f21bf66503e4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:14.138288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:14.140480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:14.262698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 539/1066] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:13:13 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/0ed16fea638a45938eee8cdf234333f1/roles/37c9983c4e5b4734a384f21bf66503e4 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:13:14.311636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc630d-5c8f-4c75-89e6-a4309d4d62af None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:14.393729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:14.425166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc630d-5c8f-4c75-89e6-a4309d4d62af None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:14.462363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:14.463018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:14.463295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:14.463562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:14.473998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:14.478711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:14.479832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:14.481086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 528/1067] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:13:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:14.486075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:14.590304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23548be-29a7-4d89-b75c-34b856c23705 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:14.625608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:14.626215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:14.626494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:14.626794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:14.627458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:14.627759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:14.629114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:14.642265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 540/1068] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:13:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:14.644028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:14.748630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba68013-187d-488a-9d67-5f6edda0796c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:14.781595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6282ca-382a-4227-b0f5-b0dd57305150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:14.782216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6282ca-382a-4227-b0f5-b0dd57305150 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:14.782489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6282ca-382a-4227-b0f5-b0dd57305150 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:14.782783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6282ca-382a-4227-b0f5-b0dd57305150 None admin] PATH_INFO: `/v3/users/ef3df0b90cfb4c308b62b209bbf3e947` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:14.785448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:14.797376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6282ca-382a-4227-b0f5-b0dd57305150 None admin] RBAC: Authorizing `identity:get_user(user_id=ef3df0b90cfb4c308b62b209bbf3e947)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:14.798826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6282ca-382a-4227-b0f5-b0dd57305150 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:14.805869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 529/1069] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:14 2022] GET /identity/v3/users/ef3df0b90cfb4c308b62b209bbf3e947 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:14.807168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:14.838800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb7ab6-78a6-4edf-8b8c-df58946df863 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:14.839077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb7ab6-78a6-4edf-8b8c-df58946df863 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:14.839349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb7ab6-78a6-4edf-8b8c-df58946df863 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:14.840797 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 530/1070] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:13:14.845406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c146e5-5f07-4624-b54f-5c826f2cc902 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:14.845681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c146e5-5f07-4624-b54f-5c826f2cc902 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:14.845948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c146e5-5f07-4624-b54f-5c826f2cc902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:14.864306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-61c146e5-5f07-4624-b54f-5c826f2cc902 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:14.865482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-61c146e5-5f07-4624-b54f-5c826f2cc902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:14.869692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:14.916645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108d5fc-783e-4221-989a-1489a8588e42 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:14.933798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 531/1071] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:14.952439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:14.953044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:14.953313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:14.953582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:15.021844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:15.023300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:15.076724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:15.125300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 541/1072] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:15.126126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c146e5-5f07-4624-b54f-5c826f2cc902 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:15.213186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:15.227098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c146e5-5f07-4624-b54f-5c826f2cc902 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:15.253681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:15.265089 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:15.265716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:15.265993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:15.270562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:15.270562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] RBAC: Authorizing `identity:get_project(project_id=c6ec40b719ff444c9a729926cb574345)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:15.270562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:15.271673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 532/1073] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:15 2022] GET /identity/v3/projects/c6ec40b719ff444c9a729926cb574345 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:15.278106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:15.319385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c53f0d-73b3-46df-aac4-aed6e894786b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:15.358285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:15.358930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:15.359209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:15.359481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:15.385121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8d9d-afed-4f24-aef4-a34767c5cf23 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:15.388710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:15.428519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:15.429657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:15.430111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:15.430565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:15.431512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=c6ec40b719ff444c9a729926cb574345, user.id=ef3df0b90cfb4c308b62b209bbf3e947) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:15.431959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:15.433552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:15.435205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:15.436799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:15.439473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 533/1074] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:15 2022] GET /identity/v3/role_assignments?user.id=ef3df0b90cfb4c308b62b209bbf3e947&scope.project.id=c6ec40b719ff444c9a729926cb574345&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:15.552822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 542/1075] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:13:15.558805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:15.619013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:15.665406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b1b4-67f0-406f-8995-100be044f626 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:15.699119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046531f-82b5-42af-a531-2b95162002ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:15.699747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046531f-82b5-42af-a531-2b95162002ad None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:15.700026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046531f-82b5-42af-a531-2b95162002ad None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:15.700297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046531f-82b5-42af-a531-2b95162002ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:15.710969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046531f-82b5-42af-a531-2b95162002ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:15.713005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046531f-82b5-42af-a531-2b95162002ad None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-1046531f-82b5-42af-a531-2b95162002ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:15.727439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:15.728418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:15.729318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:15.729318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:15.729318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:15.730966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 534/1076] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:15.734826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:15.791233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:15.836740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ac01a-1885-4533-8835-71f0003e43d4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:15.872401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a078b-6daf-4e3c-a4de-334aa70b0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:15.873393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a078b-6daf-4e3c-a4de-334aa70b0c72 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:15.873899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a078b-6daf-4e3c-a4de-334aa70b0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:15.874424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a078b-6daf-4e3c-a4de-334aa70b0c72 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:15.875388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a078b-6daf-4e3c-a4de-334aa70b0c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:15.875893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a078b-6daf-4e3c-a4de-334aa70b0c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:15.877472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a078b-6daf-4e3c-a4de-334aa70b0c72 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:15.891342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 543/1077] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:16.190403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae73b2-8bab-420c-a90d-bbaa2b2573ad None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:16.190829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae73b2-8bab-420c-a90d-bbaa2b2573ad None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:16.191108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae73b2-8bab-420c-a90d-bbaa2b2573ad None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:16.192456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 535/1078] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:16.197726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178d5ac-1679-419e-bc7e-72d95ba44698 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:16.198014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178d5ac-1679-419e-bc7e-72d95ba44698 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:16.198290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178d5ac-1679-419e-bc7e-72d95ba44698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:16.215756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a178d5ac-1679-419e-bc7e-72d95ba44698 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:16.216932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a178d5ac-1679-419e-bc7e-72d95ba44698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:16.285361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 544/1079] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:16.556822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae73b2-8bab-420c-a90d-bbaa2b2573ad None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:16.656434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae73b2-8bab-420c-a90d-bbaa2b2573ad None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:16.665654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:16.689312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:16.689930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:16.690208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:16.690475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:16.701132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:16.705725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:16.706893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:16.707821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 536/1080] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 12:13:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:16.713308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178d5ac-1679-419e-bc7e-72d95ba44698 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:16.757466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:16.808612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178d5ac-1679-419e-bc7e-72d95ba44698 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:16.841416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:16.842006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:16.842280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:16.842546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:16.843224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:16.843520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:16.844865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:16.853142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 545/1081] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 12:13:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:16.859143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:16.975833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fc28d-624d-4dd6-bf9b-1ee3007ec9ad None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.009382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.009995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.010275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.010546 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.020797 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:17.022236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:17.036634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:17.037742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:17.039050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:17.039050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:17.039050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:17.039050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 537/1082] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:17.043630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.139810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e54b82-5d29-4a33-b3a3-06b908454ae2 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.148173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:17.171986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813e455-4a29-42cc-86ca-7a4ad901b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.172570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e455-4a29-42cc-86ca-7a4ad901b12e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.172843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e455-4a29-42cc-86ca-7a4ad901b12e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.173108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e455-4a29-42cc-86ca-7a4ad901b12e None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.173819 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813e455-4a29-42cc-86ca-7a4ad901b12e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:17.174111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813e455-4a29-42cc-86ca-7a4ad901b12e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:17.175573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813e455-4a29-42cc-86ca-7a4ad901b12e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:17.187417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 546/1083] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:17.193111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.215907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:17.291291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29bcd0-44a0-461d-8876-349dcb83196f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.324618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.325236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.325514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.325786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.334138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:17.334551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36febffc-5bc5-4a30-aacb-f4f18bf4a6bf None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.334873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36febffc-5bc5-4a30-aacb-f4f18bf4a6bf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.335152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36febffc-5bc5-4a30-aacb-f4f18bf4a6bf None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.335597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:17.336525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 547/1084] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:17.341285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41293b-0590-41cc-84ab-d26c5e1f3631 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.341572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41293b-0590-41cc-84ab-d26c5e1f3631 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.341848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41293b-0590-41cc-84ab-d26c5e1f3631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:13:17.346611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:17.347543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:17.348512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 538/1085] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:17.353292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.361490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4f41293b-0590-41cc-84ab-d26c5e1f3631 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:17.362705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4f41293b-0590-41cc-84ab-d26c5e1f3631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:17.428682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 548/1086] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:17.432793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:17.451285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161fbf9-5ae4-4f26-ab80-c00b258e66f8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.483793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.484378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.484657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.484923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.485632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:17.485927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:17.487365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:17.504019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 539/1087] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:17.510478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41293b-0590-41cc-84ab-d26c5e1f3631 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.602599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:17.606727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41293b-0590-41cc-84ab-d26c5e1f3631 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.640408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.640996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.641272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.641539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.695808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.707698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:17.709138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:17.793785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36365ce-4d32-47ad-888c-9f6bee895b81 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.805461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:17.809705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 549/1088] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:13:17.815483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.827261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.827887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.828171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.828444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.835243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:17.838849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:17.843521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:17.844685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:17.845747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:17.849584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 540/1089] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:17.852084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.915308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137875c-9f73-4e93-ad81-cc58626fd6b8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.947456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be5ae1-dfe2-4637-940a-aceba5cb0cc1 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:17.948832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.949448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.949724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.950009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.960060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:17.961487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:17.975380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:17.977031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:13:17.978114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:17.978114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:13:17.978114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:17.978114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 550/1090] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:13:17.982813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:17.983432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:17.984027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:17.984306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:17.984576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:17.985242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:17.985540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:17.986932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:17.995852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 541/1091] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:18.001834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:18.033816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:18.058909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:18.083148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1935d-d5fd-40f9-8ada-032ea438aad6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:18.098429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9a7f6-860c-43ee-9bd1-340a89cea710 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:18.116986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:18.117579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:18.117856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:18.118128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:18.118794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:18.119101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:18.120492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:18.136539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 551/1092] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:18.136684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:18.136684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:18.136684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:18.136684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] PATH_INFO: `/v3/users/ef3df0b90cfb4c308b62b209bbf3e947` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:18.137106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] RBAC: Authorizing `identity:get_user(user_id=ef3df0b90cfb4c308b62b209bbf3e947)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:18.137106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:18.137526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:18.137965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 542/1093] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:18 2022] GET /identity/v3/users/ef3df0b90cfb4c308b62b209bbf3e947 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:18.143494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:18.235995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ead6e-05c9-4545-a67d-ca62888a5c28 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:18.243548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723143c5-75ad-4ed0-80b7-9e630bde738a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:18.247173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:18.269223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:18.271032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:18.271651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:18.271924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:18.272191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:18.278102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:18.278742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:18.279026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:18.279300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:18.279877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:18.280565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] RBAC: Authorizing `identity:get_project(project_id=c6ec40b719ff444c9a729926cb574345)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:18.281309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:18.282026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:18.283887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 543/1094] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:18 2022] GET /identity/v3/projects/c6ec40b719ff444c9a729926cb574345 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:18.290147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:13:18.297183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:18.298050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:18.299067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 552/1095] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:13:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:18.304272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:18.392330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b6313-2ea5-4797-ae4c-4f798c6bac9d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:18.401186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1877b-9e8b-4a28-a095-3c7de1455c31 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:18.429556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fe994-87ef-4463-9c6b-f7fc9d2b78f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:18.430242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fe994-87ef-4463-9c6b-f7fc9d2b78f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:18.430519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fe994-87ef-4463-9c6b-f7fc9d2b78f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:18.430826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fe994-87ef-4463-9c6b-f7fc9d2b78f6 None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345/users/ef3df0b90cfb4c308b62b209bbf3e947/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:18.432812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fe994-87ef-4463-9c6b-f7fc9d2b78f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6ec40b719ff444c9a729926cb574345, user_id=ef3df0b90cfb4c308b62b209bbf3e947, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:18.434229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fe994-87ef-4463-9c6b-f7fc9d2b78f6 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:18.437443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:18.437673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:18.437950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:18.438215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:18.438932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:18.439232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:18.440655 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:18.452696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 553/1096] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:18.460060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:18.490976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:18.501422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 544/1097] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:13:18 2022] PUT /identity/v3/projects/c6ec40b719ff444c9a729926cb574345/users/ef3df0b90cfb4c308b62b209bbf3e947/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:13:18.564652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d019b9-870c-422d-ae51-9705ff22cde8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:18.600257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397fe93-816f-4aec-9c9f-3802e7dc6c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:18.600889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fe93-816f-4aec-9c9f-3802e7dc6c88 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:18.601170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fe93-816f-4aec-9c9f-3802e7dc6c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:18.601428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397fe93-816f-4aec-9c9f-3802e7dc6c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:18.602203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397fe93-816f-4aec-9c9f-3802e7dc6c88 None admin] RBAC: Adding query filter params (role.id=37c9983c4e5b4734a384f21bf66503e4, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=0ed16fea638a45938eee8cdf234333f1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:18.602508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397fe93-816f-4aec-9c9f-3802e7dc6c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:18.603990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397fe93-816f-4aec-9c9f-3802e7dc6c88 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:18.610754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 554/1098] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:18 2022] GET /identity/v3/role_assignments?user.id=0ed16fea638a45938eee8cdf234333f1&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=37c9983c4e5b4734a384f21bf66503e4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:20.408886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de535-8cee-4203-a913-ba7fc211591e None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:20.409408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de535-8cee-4203-a913-ba7fc211591e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:20.409408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2de535-8cee-4203-a913-ba7fc211591e None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:20.410800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 545/1099] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:20.416159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83766432-3b68-4478-95c0-b6622e8a7a2e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:20.416585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83766432-3b68-4478-95c0-b6622e8a7a2e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:20.416700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83766432-3b68-4478-95c0-b6622e8a7a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:20.434983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-83766432-3b68-4478-95c0-b6622e8a7a2e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:20.436201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-83766432-3b68-4478-95c0-b6622e8a7a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:20.500474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 555/1100] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:20.765018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de535-8cee-4203-a913-ba7fc211591e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:20.820608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:20.863455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2de535-8cee-4203-a913-ba7fc211591e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:20.896623 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:20.897232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:20.897513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:20.897803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:20.908162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:20.912997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:20.914618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:20.915716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 546/1101] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:13:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:20.920552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83766432-3b68-4478-95c0-b6622e8a7a2e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:20.944846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:21.016279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83766432-3b68-4478-95c0-b6622e8a7a2e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:21.120186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:21.120722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:21.121001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:21.121274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:21.121930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:21.122234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:21.123666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:21.131476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 556/1102] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:13:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:21.137281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:21.234773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db290733-14ab-4cca-9ee1-091a6542ba67 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:21.250046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:21.266982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:21.267563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:21.267842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:21.268115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] PATH_INFO: `/v3/users/ef3df0b90cfb4c308b62b209bbf3e947` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:21.269226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] RBAC: Authorizing `identity:get_user(user_id=ef3df0b90cfb4c308b62b209bbf3e947)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:21.270667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:21.272313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 547/1103] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:21 2022] GET /identity/v3/users/ef3df0b90cfb4c308b62b209bbf3e947 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:21.278841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:21.362118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:21.376289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261237b-4fcc-4c26-a15a-abca169a67a7 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:21.409385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:21.409977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:21.410255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:21.410526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:21.476397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:21.477848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:21.557797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:21.572691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 557/1104] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:21.578846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:21.674233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc384fc-781d-4a12-ac4b-02f4c5fd49c1 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:21.706803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11453e1-e22d-4e0e-ba32-5724c1532f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:21.707406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11453e1-e22d-4e0e-ba32-5724c1532f10 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:21.707683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11453e1-e22d-4e0e-ba32-5724c1532f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:21.707953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11453e1-e22d-4e0e-ba32-5724c1532f10 None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:21.709042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11453e1-e22d-4e0e-ba32-5724c1532f10 None admin] RBAC: Authorizing `identity:get_project(project_id=c6ec40b719ff444c9a729926cb574345)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:21.710455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11453e1-e22d-4e0e-ba32-5724c1532f10 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:21.712691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 548/1105] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:21 2022] GET /identity/v3/projects/c6ec40b719ff444c9a729926cb574345 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:21.718780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:21.813938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a695e-341e-468e-b53b-e043a1a0a760 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:21.846212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7300592-20bd-4c5d-b7c2-c8d438d468e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:21.846820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7300592-20bd-4c5d-b7c2-c8d438d468e3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:21.847098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7300592-20bd-4c5d-b7c2-c8d438d468e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:21.847362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7300592-20bd-4c5d-b7c2-c8d438d468e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:21.848071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7300592-20bd-4c5d-b7c2-c8d438d468e3 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=c6ec40b719ff444c9a729926cb574345, user.id=ef3df0b90cfb4c308b62b209bbf3e947) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:21.848372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7300592-20bd-4c5d-b7c2-c8d438d468e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:21.849768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7300592-20bd-4c5d-b7c2-c8d438d468e3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:21.855327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 558/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:21 2022] GET /identity/v3/role_assignments?user.id=ef3df0b90cfb4c308b62b209bbf3e947&scope.project.id=c6ec40b719ff444c9a729926cb574345&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:23.786617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267b6-bb30-48f0-9bb7-68ba4b46f932 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:23.787176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267b6-bb30-48f0-9bb7-68ba4b46f932 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:23.787176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267b6-bb30-48f0-9bb7-68ba4b46f932 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:23.788534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 549/1107] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:23.794366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1551af-770b-4007-9d74-a8c6ecfcbfdf None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:23.794807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1551af-770b-4007-9d74-a8c6ecfcbfdf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:23.794959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1551af-770b-4007-9d74-a8c6ecfcbfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:23.813070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5d1551af-770b-4007-9d74-a8c6ecfcbfdf None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:23.814252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5d1551af-770b-4007-9d74-a8c6ecfcbfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:23.858850 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:23.878768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 559/1108] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:24.152973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef267b6-bb30-48f0-9bb7-68ba4b46f932 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:24.206164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:24.252083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef267b6-bb30-48f0-9bb7-68ba4b46f932 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:24.283956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157f698-c5e4-457e-8591-fc6594e9db7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:24.284559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157f698-c5e4-457e-8591-fc6594e9db7f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:24.284836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157f698-c5e4-457e-8591-fc6594e9db7f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:24.285106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157f698-c5e4-457e-8591-fc6594e9db7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:24.286191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157f698-c5e4-457e-8591-fc6594e9db7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:24.287636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157f698-c5e4-457e-8591-fc6594e9db7f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:24.289200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 550/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:24.294853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1551af-770b-4007-9d74-a8c6ecfcbfdf None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:24.392005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1551af-770b-4007-9d74-a8c6ecfcbfdf None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:24.424806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5179c-5270-43c6-ab10-14430e01392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:24.425332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5179c-5270-43c6-ab10-14430e01392d None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:24.425611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5179c-5270-43c6-ab10-14430e01392d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:24.425881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5179c-5270-43c6-ab10-14430e01392d None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:24.426518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5179c-5270-43c6-ab10-14430e01392d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:24.427974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5179c-5270-43c6-ab10-14430e01392d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:24.541928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 560/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:13:24 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:26.495475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79600-ad83-45e5-905b-f968493a9238 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:26.495964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79600-ad83-45e5-905b-f968493a9238 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:26.496073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce79600-ad83-45e5-905b-f968493a9238 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:26.497348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 551/1111] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:26.502886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222e3c4-3710-4554-9dc0-f79178c5f046 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:26.503254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222e3c4-3710-4554-9dc0-f79178c5f046 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:26.503439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222e3c4-3710-4554-9dc0-f79178c5f046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:26.521618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0222e3c4-3710-4554-9dc0-f79178c5f046 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:26.522842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0222e3c4-3710-4554-9dc0-f79178c5f046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:26.550497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:26.588311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 561/1112] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:26.899954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79600-ad83-45e5-905b-f968493a9238 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:27.000736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce79600-ad83-45e5-905b-f968493a9238 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:27.019538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:27.032605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:27.033215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:27.033492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:27.033764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] PATH_INFO: `/v3/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:27.035185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] RBAC: Authorizing `identity:get_role(role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:27.036536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:27.037892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 552/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:26 2022] GET /identity/v3/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:27.044453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222e3c4-3710-4554-9dc0-f79178c5f046 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:27.155119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222e3c4-3710-4554-9dc0-f79178c5f046 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:27.187764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:27.188377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:27.188657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:27.188929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] PATH_INFO: `/v3/users/d90f2a4592e5405aa8b4dead5952f906` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:27.218989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] RBAC: Authorizing `identity:get_user(user_id=d90f2a4592e5405aa8b4dead5952f906)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:27.220409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:27.221998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 562/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:27 2022] GET /identity/v3/users/d90f2a4592e5405aa8b4dead5952f906 => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:27.228809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:27.324451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000e3a8-68c0-48c6-99fc-a0654ff2676d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:27.357191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:27.357777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:27.358054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:27.358325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:27.477449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:27.489486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:27.491069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:27.584581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 553/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:27.591422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:27.615921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:27.687574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37176b4-ba32-403e-8264-da8306f7a4aa None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:27.722024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7e6bd-bce0-4e6a-a49a-ab8ff8dfdbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:27.722637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e6bd-bce0-4e6a-a49a-ab8ff8dfdbec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:27.722944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e6bd-bce0-4e6a-a49a-ab8ff8dfdbec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:27.723217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7e6bd-bce0-4e6a-a49a-ab8ff8dfdbec None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:27.724314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7e6bd-bce0-4e6a-a49a-ab8ff8dfdbec None admin] RBAC: Authorizing `identity:get_project(project_id=c6ec40b719ff444c9a729926cb574345)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:27.725742 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7e6bd-bce0-4e6a-a49a-ab8ff8dfdbec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:27.727372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 563/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:27 2022] GET /identity/v3/projects/c6ec40b719ff444c9a729926cb574345 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:27.733803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:27.816846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:27.828728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a37c-acaf-4276-879e-0b6fb00f3ff1 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:27.860992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633806a-80fc-4ebf-882f-67fdd8e51eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:27.861577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633806a-80fc-4ebf-882f-67fdd8e51eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:27.861855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633806a-80fc-4ebf-882f-67fdd8e51eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:27.862126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633806a-80fc-4ebf-882f-67fdd8e51eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:27.862867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633806a-80fc-4ebf-882f-67fdd8e51eb9 None admin] RBAC: Adding query filter params (role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=c6ec40b719ff444c9a729926cb574345, user.id=d90f2a4592e5405aa8b4dead5952f906) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:27.863196 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633806a-80fc-4ebf-882f-67fdd8e51eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:27.864614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633806a-80fc-4ebf-882f-67fdd8e51eb9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:27.870323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 554/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:27 2022] GET /identity/v3/role_assignments?user.id=d90f2a4592e5405aa8b4dead5952f906&scope.project.id=c6ec40b719ff444c9a729926cb574345&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:29.724379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408bfef-715a-4869-bb16-c0d5c4279341 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:29.724922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408bfef-715a-4869-bb16-c0d5c4279341 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:29.724922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408bfef-715a-4869-bb16-c0d5c4279341 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:29.726165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 564/1118] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:29.731997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa15d4-b630-4761-873b-090c357b8759 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:29.732390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa15d4-b630-4761-873b-090c357b8759 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:29.732537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa15d4-b630-4761-873b-090c357b8759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:29.751165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-4daa15d4-b630-4761-873b-090c357b8759 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:29.752336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4daa15d4-b630-4761-873b-090c357b8759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:29.815770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 555/1119] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:30.078732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408bfef-715a-4869-bb16-c0d5c4279341 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:30.166894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:30.175827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408bfef-715a-4869-bb16-c0d5c4279341 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:30.208206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:30.208812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:30.209088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:30.209360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] PATH_INFO: `/v3/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:30.211267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] RBAC: Authorizing `identity:get_role(role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:30.212686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:30.213972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 565/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:30 2022] GET /identity/v3/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:30.219656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa15d4-b630-4761-873b-090c357b8759 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:30.296619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:30.315168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa15d4-b630-4761-873b-090c357b8759 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:30.348492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:30.349042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:30.349313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:30.349580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] PATH_INFO: `/v3/users/d90f2a4592e5405aa8b4dead5952f906` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:30.350685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] RBAC: Authorizing `identity:get_user(user_id=d90f2a4592e5405aa8b4dead5952f906)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:30.352094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:30.353637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 556/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:30 2022] GET /identity/v3/users/d90f2a4592e5405aa8b4dead5952f906 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:13:30.359714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:30.456411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6d174-f67b-41fc-8d09-572a6992a33e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:30.489238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1014ad-1a92-4468-b0e3-ca1ea09c88fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:30.489842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1014ad-1a92-4468-b0e3-ca1ea09c88fe None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:30.490120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1014ad-1a92-4468-b0e3-ca1ea09c88fe None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:30.490392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1014ad-1a92-4468-b0e3-ca1ea09c88fe None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:30.491508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1014ad-1a92-4468-b0e3-ca1ea09c88fe None admin] RBAC: Authorizing `identity:get_project(project_id=c6ec40b719ff444c9a729926cb574345)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:30.493500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1014ad-1a92-4468-b0e3-ca1ea09c88fe None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:30.495141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 566/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:30 2022] GET /identity/v3/projects/c6ec40b719ff444c9a729926cb574345 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:30.501180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:30.594982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ad783-f908-40fc-9bb3-1a9fbed2c5bd None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:30.627983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b58c6-6282-4404-b17d-70dd23ee882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:30.628559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b58c6-6282-4404-b17d-70dd23ee882f None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:30.628835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b58c6-6282-4404-b17d-70dd23ee882f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:30.629105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b58c6-6282-4404-b17d-70dd23ee882f None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345/users/d90f2a4592e5405aa8b4dead5952f906/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:30.631211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b58c6-6282-4404-b17d-70dd23ee882f None admin] RBAC: Authorizing `identity:create_grant(project_id=c6ec40b719ff444c9a729926cb574345, user_id=d90f2a4592e5405aa8b4dead5952f906, role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:30.632651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b58c6-6282-4404-b17d-70dd23ee882f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:30.735083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 557/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:13:30 2022] PUT /identity/v3/projects/c6ec40b719ff444c9a729926cb574345/users/d90f2a4592e5405aa8b4dead5952f906/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:13:32.598734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac504c-d10e-4eeb-8ca6-b19899852255 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:32.599271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac504c-d10e-4eeb-8ca6-b19899852255 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:32.599271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac504c-d10e-4eeb-8ca6-b19899852255 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:32.600590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 567/1124] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:32.605728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121c683-1d8b-4a9f-a5db-895a9d6387d4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:32.606417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121c683-1d8b-4a9f-a5db-895a9d6387d4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:32.606596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121c683-1d8b-4a9f-a5db-895a9d6387d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:32.624397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5121c683-1d8b-4a9f-a5db-895a9d6387d4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:32.625558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5121c683-1d8b-4a9f-a5db-895a9d6387d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:32.628229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:32.689352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 558/1125] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:32.955681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac504c-d10e-4eeb-8ca6-b19899852255 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:32.983411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:33.055044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac504c-d10e-4eeb-8ca6-b19899852255 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:33.087386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:33.087982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:33.088254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:33.088522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] PATH_INFO: `/v3/roles/efc9b08e5a7446b19b5245cfa60e9893` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:33.089872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] RBAC: Authorizing `identity:get_role(role_id=efc9b08e5a7446b19b5245cfa60e9893)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:33.091259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:33.092538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 568/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:32 2022] GET /identity/v3/roles/efc9b08e5a7446b19b5245cfa60e9893 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:33.098237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121c683-1d8b-4a9f-a5db-895a9d6387d4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:33.193325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121c683-1d8b-4a9f-a5db-895a9d6387d4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:33.225135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:33.225683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:33.225955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:33.226840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] PATH_INFO: `/v3/users/d90f2a4592e5405aa8b4dead5952f906` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:33.227943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] RBAC: Authorizing `identity:get_user(user_id=d90f2a4592e5405aa8b4dead5952f906)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:33.229358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:33.231226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 559/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:33 2022] GET /identity/v3/users/d90f2a4592e5405aa8b4dead5952f906 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:33.236973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:33.326104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:33.331612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502c38d-4ec4-4f86-b33e-9316d9923dec None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:33.364880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:33.365462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:33.365741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:33.366011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:33.430066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:33.431566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:33.519581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:33.526371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 569/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:33.532273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:33.542776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:33.625960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd941e7a-0bc5-47ba-b313-2b51cef66870 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:33.658269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0e6de-3475-4933-aea1-9894bdff0807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:33.658893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0e6de-3475-4933-aea1-9894bdff0807 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:33.659168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0e6de-3475-4933-aea1-9894bdff0807 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:33.659435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0e6de-3475-4933-aea1-9894bdff0807 None admin] PATH_INFO: `/v3/projects/c6ec40b719ff444c9a729926cb574345` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:33.660511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0e6de-3475-4933-aea1-9894bdff0807 None admin] RBAC: Authorizing `identity:get_project(project_id=c6ec40b719ff444c9a729926cb574345)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:33.661921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0e6de-3475-4933-aea1-9894bdff0807 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:33.663545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 560/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:33 2022] GET /identity/v3/projects/c6ec40b719ff444c9a729926cb574345 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:33.669597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:33.763575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c60f-d2a3-4a8a-a5a9-0ddb5ef084ee None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:33.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cea5c6-af36-411a-ae02-0676e17e8289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:33.796372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cea5c6-af36-411a-ae02-0676e17e8289 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:33.796648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cea5c6-af36-411a-ae02-0676e17e8289 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:33.796914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cea5c6-af36-411a-ae02-0676e17e8289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:33.797619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cea5c6-af36-411a-ae02-0676e17e8289 None admin] RBAC: Adding query filter params (role.id=efc9b08e5a7446b19b5245cfa60e9893, scope.project.id=c6ec40b719ff444c9a729926cb574345, user.id=d90f2a4592e5405aa8b4dead5952f906) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:33.797917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cea5c6-af36-411a-ae02-0676e17e8289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:33.799496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cea5c6-af36-411a-ae02-0676e17e8289 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:33.805139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 570/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:33 2022] GET /identity/v3/role_assignments?user.id=d90f2a4592e5405aa8b4dead5952f906&scope.project.id=c6ec40b719ff444c9a729926cb574345&role.id=efc9b08e5a7446b19b5245cfa60e9893 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:35.704007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6e794-838d-40ee-9e49-c6c2f372fd50 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:35.704495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6e794-838d-40ee-9e49-c6c2f372fd50 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:35.704495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6e794-838d-40ee-9e49-c6c2f372fd50 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:35.705799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 561/1131] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:35.710934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cbae3-278a-4234-b2bf-05879a6c0b86 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:35.711312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cbae3-278a-4234-b2bf-05879a6c0b86 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:35.711480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cbae3-278a-4234-b2bf-05879a6c0b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:35.729471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-406cbae3-278a-4234-b2bf-05879a6c0b86 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:35.730643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-406cbae3-278a-4234-b2bf-05879a6c0b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:35.784061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:35.794950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 571/1132] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:36.059684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6e794-838d-40ee-9e49-c6c2f372fd50 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:36.135068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:36.158172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6e794-838d-40ee-9e49-c6c2f372fd50 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:36.191417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9ff7c-2a49-4c1c-b554-b739d5fe9661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:36.192024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ff7c-2a49-4c1c-b554-b739d5fe9661 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:36.192304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ff7c-2a49-4c1c-b554-b739d5fe9661 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:36.192577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ff7c-2a49-4c1c-b554-b739d5fe9661 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:36.193672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9ff7c-2a49-4c1c-b554-b739d5fe9661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:36.195133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9ff7c-2a49-4c1c-b554-b739d5fe9661 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:36.196712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 562/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:36.202400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cbae3-278a-4234-b2bf-05879a6c0b86 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:36.299907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cbae3-278a-4234-b2bf-05879a6c0b86 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:36.331974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fddeb-19c9-4b9a-a2c7-01dd50b719d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:36.332502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fddeb-19c9-4b9a-a2c7-01dd50b719d7 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:36.332782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fddeb-19c9-4b9a-a2c7-01dd50b719d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:36.333053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fddeb-19c9-4b9a-a2c7-01dd50b719d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:36.333682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fddeb-19c9-4b9a-a2c7-01dd50b719d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:36.335134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fddeb-19c9-4b9a-a2c7-01dd50b719d7 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:36.432242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 572/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:13:36 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:38.326276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c207e-4f27-4c27-b304-0c664f7e4837 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:38.326828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c207e-4f27-4c27-b304-0c664f7e4837 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:38.326828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c207e-4f27-4c27-b304-0c664f7e4837 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:38.328174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 563/1135] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:38.333409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded10e-24f0-4d61-90aa-c22d1ce625b1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:38.333824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded10e-24f0-4d61-90aa-c22d1ce625b1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:38.333942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ded10e-24f0-4d61-90aa-c22d1ce625b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:38.351685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b5ded10e-24f0-4d61-90aa-c22d1ce625b1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:38.352864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b5ded10e-24f0-4d61-90aa-c22d1ce625b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:38.387520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:38.417024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 573/1136] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:38.684818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c207e-4f27-4c27-b304-0c664f7e4837 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:38.780004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c207e-4f27-4c27-b304-0c664f7e4837 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:38.812892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de125718-45cf-4700-b079-ff242db53056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:38.813491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de125718-45cf-4700-b079-ff242db53056 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:38.813765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de125718-45cf-4700-b079-ff242db53056 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:38.814028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de125718-45cf-4700-b079-ff242db53056 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:38.815127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de125718-45cf-4700-b079-ff242db53056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:38.816526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de125718-45cf-4700-b079-ff242db53056 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:38.818040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 564/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:38.823620 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ded10e-24f0-4d61-90aa-c22d1ce625b1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:38.918338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ded10e-24f0-4d61-90aa-c22d1ce625b1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:38.950617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d08643-2cd7-4d8a-ba3f-bf52d39a4554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:38.951173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d08643-2cd7-4d8a-ba3f-bf52d39a4554 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:38.951448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d08643-2cd7-4d8a-ba3f-bf52d39a4554 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:38.951711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d08643-2cd7-4d8a-ba3f-bf52d39a4554 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:38.952336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d08643-2cd7-4d8a-ba3f-bf52d39a4554 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:38.953750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d08643-2cd7-4d8a-ba3f-bf52d39a4554 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:39.083291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 574/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:13:38 2022] POST /identity/v3/users => generated 286 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:41.001809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a97e-1823-4e81-8f31-d3f9a7fb7138 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:41.002338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a97e-1823-4e81-8f31-d3f9a7fb7138 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:41.002338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9a97e-1823-4e81-8f31-d3f9a7fb7138 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:41.003719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 565/1139] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:41.009481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8154-b2ba-402b-bf61-9e8331b4b312 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:41.009863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8154-b2ba-402b-bf61-9e8331b4b312 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:41.010016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8154-b2ba-402b-bf61-9e8331b4b312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:41.028002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e56c8154-b2ba-402b-bf61-9e8331b4b312 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:41.029175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e56c8154-b2ba-402b-bf61-9e8331b4b312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:41.051622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:41.093425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 575/1140] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:41.361541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9a97e-1823-4e81-8f31-d3f9a7fb7138 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:41.376025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:41.460188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9a97e-1823-4e81-8f31-d3f9a7fb7138 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:41.493747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:41.494357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:41.494636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:41.494937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:41.504727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:41.509390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:41.510569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:41.511710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 566/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:41.516708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8154-b2ba-402b-bf61-9e8331b4b312 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:41.612578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8154-b2ba-402b-bf61-9e8331b4b312 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:41.645404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:41.645987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:41.646265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:41.646536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:41.647220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:41.647522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:41.649431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:41.657216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 576/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:41.662868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:41.731575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:41.759169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be993e6-5a85-421c-8f8f-ad72d5672bdf None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:41.792715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:41.793319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:41.793596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:41.793869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] PATH_INFO: `/v3/users/9055dba8b80f47e8aab573717d99c44f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:41.806985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] RBAC: Authorizing `identity:get_user(user_id=9055dba8b80f47e8aab573717d99c44f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:41.808413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:41.809984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 567/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:41 2022] GET /identity/v3/users/9055dba8b80f47e8aab573717d99c44f => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:41.815793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:41.837258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:41.913025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06772e2b-af56-450b-aac8-392baf635ae8 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:41.946566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:41.947178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:41.947454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:41.947721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:42.013242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:42.014693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:42.034619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:42.109831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 577/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:42.115720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:42.212083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6c97b-28c5-4633-9c21-68481658c4fb None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:42.239350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:42.245485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76ff5a-96be-4240-8e53-c8b42537a59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:42.254224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ff5a-96be-4240-8e53-c8b42537a59a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:42.254224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ff5a-96be-4240-8e53-c8b42537a59a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:42.254224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ff5a-96be-4240-8e53-c8b42537a59a None admin] PATH_INFO: `/v3/projects/ad21be24769449d2920d3f75ab72ee62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:42.254224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76ff5a-96be-4240-8e53-c8b42537a59a None admin] RBAC: Authorizing `identity:get_project(project_id=ad21be24769449d2920d3f75ab72ee62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:42.254224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76ff5a-96be-4240-8e53-c8b42537a59a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:42.254224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 568/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:42 2022] GET /identity/v3/projects/ad21be24769449d2920d3f75ab72ee62 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:42.256973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:42.352457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7dd8d-41a0-4413-90d3-588f36189c7a None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:42.371714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:42.385376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb6915-e469-45a9-ba77-ea437ee50986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:42.385956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6915-e469-45a9-ba77-ea437ee50986 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:42.386232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6915-e469-45a9-ba77-ea437ee50986 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:42.386501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6915-e469-45a9-ba77-ea437ee50986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:42.387245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6915-e469-45a9-ba77-ea437ee50986 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=ad21be24769449d2920d3f75ab72ee62, user.id=9055dba8b80f47e8aab573717d99c44f) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:42.387548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6915-e469-45a9-ba77-ea437ee50986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:42.388956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6915-e469-45a9-ba77-ea437ee50986 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:42.394067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 578/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:42 2022] GET /identity/v3/role_assignments?user.id=9055dba8b80f47e8aab573717d99c44f&scope.project.id=ad21be24769449d2920d3f75ab72ee62&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:44.254637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a3d1-ee3e-4641-b45e-548f9306a21d None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:44.255185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a3d1-ee3e-4641-b45e-548f9306a21d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:44.255299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77a3d1-ee3e-4641-b45e-548f9306a21d None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:44.256561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 569/1147] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:44.262335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e4f9f-8ae7-4344-8751-d04464c080b8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:44.262791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e4f9f-8ae7-4344-8751-d04464c080b8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:44.262909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e4f9f-8ae7-4344-8751-d04464c080b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:44.281095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-046e4f9f-8ae7-4344-8751-d04464c080b8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:44.282274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-046e4f9f-8ae7-4344-8751-d04464c080b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:44.346965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 579/1148] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:44.614083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77a3d1-ee3e-4641-b45e-548f9306a21d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:44.710865 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77a3d1-ee3e-4641-b45e-548f9306a21d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:44.743647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:44.744243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:44.744517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:44.744780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:44.754304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:44.758910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:44.760002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:44.760872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 570/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:44.765770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e4f9f-8ae7-4344-8751-d04464c080b8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:44.861537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e4f9f-8ae7-4344-8751-d04464c080b8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:44.875449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:44.894197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:44.894813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:44.895092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:44.895359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:44.896010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:44.896311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:44.897661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:44.905345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 580/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:13:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:44.910937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:44.968219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:45.008357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9bef-1c4a-4260-8f3e-c5179c41298f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:45.041104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:45.041703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:45.041978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:45.042245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] PATH_INFO: `/v3/users/9055dba8b80f47e8aab573717d99c44f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:45.043350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] RBAC: Authorizing `identity:get_user(user_id=9055dba8b80f47e8aab573717d99c44f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:45.044863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:45.046396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 571/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:44 2022] GET /identity/v3/users/9055dba8b80f47e8aab573717d99c44f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:45.052086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:45.147066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89e6d9-ede4-48ad-98d2-bdc8f82f5bbb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:45.179684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88188159-4b9f-4f7e-871d-fea3e765808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:45.180285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88188159-4b9f-4f7e-871d-fea3e765808a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:45.180564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88188159-4b9f-4f7e-871d-fea3e765808a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:45.180837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88188159-4b9f-4f7e-871d-fea3e765808a None admin] PATH_INFO: `/v3/projects/ad21be24769449d2920d3f75ab72ee62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:45.181939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88188159-4b9f-4f7e-871d-fea3e765808a None admin] RBAC: Authorizing `identity:get_project(project_id=ad21be24769449d2920d3f75ab72ee62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:45.183389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88188159-4b9f-4f7e-871d-fea3e765808a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:45.184968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 581/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:45 2022] GET /identity/v3/projects/ad21be24769449d2920d3f75ab72ee62 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:45.190695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:45.286806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537718e0-8cae-4bd4-ac91-096fca913722 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:45.311647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:45.320043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10724dd1-719e-44c6-afd7-2c874c38d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:45.320715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10724dd1-719e-44c6-afd7-2c874c38d79c None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:45.323000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10724dd1-719e-44c6-afd7-2c874c38d79c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:45.323000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10724dd1-719e-44c6-afd7-2c874c38d79c None admin] PATH_INFO: `/v3/projects/ad21be24769449d2920d3f75ab72ee62/users/9055dba8b80f47e8aab573717d99c44f/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:45.323164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10724dd1-719e-44c6-afd7-2c874c38d79c None admin] RBAC: Authorizing `identity:create_grant(project_id=ad21be24769449d2920d3f75ab72ee62, user_id=9055dba8b80f47e8aab573717d99c44f, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:45.324571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10724dd1-719e-44c6-afd7-2c874c38d79c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:45.420349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 572/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:13:45 2022] PUT /identity/v3/projects/ad21be24769449d2920d3f75ab72ee62/users/9055dba8b80f47e8aab573717d99c44f/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:13:47.286036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b942c91-e099-4ea7-8c97-04b955dc5d9d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:47.286562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b942c91-e099-4ea7-8c97-04b955dc5d9d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:47.286562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b942c91-e099-4ea7-8c97-04b955dc5d9d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:47.287925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 582/1154] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:47.293097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73893-ed11-4461-a191-09e2f246a88a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:47.293466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73893-ed11-4461-a191-09e2f246a88a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:47.293644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a73893-ed11-4461-a191-09e2f246a88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:47.312501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d2a73893-ed11-4461-a191-09e2f246a88a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:47.313689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d2a73893-ed11-4461-a191-09e2f246a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:47.377599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 573/1155] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:47.646733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b942c91-e099-4ea7-8c97-04b955dc5d9d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:47.692897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:47.746989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b942c91-e099-4ea7-8c97-04b955dc5d9d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:47.778667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:47.779292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:47.779568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:47.779839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:47.790119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:13:47.794797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:47.795856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:13:47.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 583/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:47.801828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a73893-ed11-4461-a191-09e2f246a88a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:47.901427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a73893-ed11-4461-a191-09e2f246a88a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:47.917495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:47.934948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:47.935578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:47.935853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:47.936120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:47.936768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:47.937062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:47.938410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:47.946451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 574/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:47.952324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:48.050246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d64717-e5d6-4598-9f29-c70f9144e1e4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:48.052492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:48.082572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:48.083349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:48.083629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:48.083900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] PATH_INFO: `/v3/users/9055dba8b80f47e8aab573717d99c44f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:48.085004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] RBAC: Authorizing `identity:get_user(user_id=9055dba8b80f47e8aab573717d99c44f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:48.086942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:48.088494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 584/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:13:47 2022] GET /identity/v3/users/9055dba8b80f47e8aab573717d99c44f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:48.094347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:48.189517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bcaa9-607d-4d08-ab67-3a6e5ba073f8 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:48.221330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:48.221908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:48.222184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:48.222451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:48.265688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:48.286408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:48.287857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:48.383471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 575/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:48.389356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:48.486480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9470f8-3f38-4446-9999-2fa9011d97fa None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:48.519056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8fe2-6667-43ef-9fff-9f5a65f0e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:48.519663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8fe2-6667-43ef-9fff-9f5a65f0e864 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:48.519935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8fe2-6667-43ef-9fff-9f5a65f0e864 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:48.520202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8fe2-6667-43ef-9fff-9f5a65f0e864 None admin] PATH_INFO: `/v3/projects/ad21be24769449d2920d3f75ab72ee62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:48.521287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8fe2-6667-43ef-9fff-9f5a65f0e864 None admin] RBAC: Authorizing `identity:get_project(project_id=ad21be24769449d2920d3f75ab72ee62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:48.522729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8fe2-6667-43ef-9fff-9f5a65f0e864 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:48.524908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 585/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:13:48 2022] GET /identity/v3/projects/ad21be24769449d2920d3f75ab72ee62 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:48.530969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:48.604171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:48.625258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5b052-0544-4d28-a587-22386876f096 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:48.658373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def476ff-fcb3-4f89-8503-6bb71da13a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:48.658978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def476ff-fcb3-4f89-8503-6bb71da13a1c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:48.659250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def476ff-fcb3-4f89-8503-6bb71da13a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:48.659513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def476ff-fcb3-4f89-8503-6bb71da13a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:48.660217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def476ff-fcb3-4f89-8503-6bb71da13a1c None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=ad21be24769449d2920d3f75ab72ee62, user.id=9055dba8b80f47e8aab573717d99c44f) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:48.660515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def476ff-fcb3-4f89-8503-6bb71da13a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:48.661922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def476ff-fcb3-4f89-8503-6bb71da13a1c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:48.667392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 576/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:13:48 2022] GET /identity/v3/role_assignments?user.id=9055dba8b80f47e8aab573717d99c44f&scope.project.id=ad21be24769449d2920d3f75ab72ee62&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:50.585052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa73e-0917-4044-8d64-91c6572ab520 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:50.586066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa73e-0917-4044-8d64-91c6572ab520 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:50.586066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa73e-0917-4044-8d64-91c6572ab520 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:50.587422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 586/1162] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:50.592646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fc3b-5009-4961-bc93-3ab9f738110b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:50.593072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fc3b-5009-4961-bc93-3ab9f738110b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:50.593194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452fc3b-5009-4961-bc93-3ab9f738110b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:50.611443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3452fc3b-5009-4961-bc93-3ab9f738110b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:50.612601 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3452fc3b-5009-4961-bc93-3ab9f738110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:50.675123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 577/1163] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:50.942403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa73e-0917-4044-8d64-91c6572ab520 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:50.985340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:51.041169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa73e-0917-4044-8d64-91c6572ab520 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:51.073583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:51.074155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:51.074468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:51.074724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:51.138558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:51.140116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:51.177364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:51.233877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 587/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:13:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:13:51.239705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452fc3b-5009-4961-bc93-3ab9f738110b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:51.305997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:51.334037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452fc3b-5009-4961-bc93-3ab9f738110b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:51.368032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d038e4-3f67-4106-8749-ab7ee8cc8ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:51.368734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d038e4-3f67-4106-8749-ab7ee8cc8ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:51.369015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d038e4-3f67-4106-8749-ab7ee8cc8ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:51.369286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d038e4-3f67-4106-8749-ab7ee8cc8ef9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:51.375876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d038e4-3f67-4106-8749-ab7ee8cc8ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:51.377306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d038e4-3f67-4106-8749-ab7ee8cc8ef9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-06d038e4-3f67-4106-8749-ab7ee8cc8ef9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:13:51.388844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:13:51.394376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:13:51.395363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 578/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:13:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:13:51.396176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:51.492960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e897db0-2aec-4c97-96c0-41c659a7e248 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:51.525323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8de54-735a-419a-af7a-26db234eacaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:51.525911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8de54-735a-419a-af7a-26db234eacaa None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:51.526190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8de54-735a-419a-af7a-26db234eacaa None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:51.526461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8de54-735a-419a-af7a-26db234eacaa None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:51.527122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8de54-735a-419a-af7a-26db234eacaa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:13:51.527426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8de54-735a-419a-af7a-26db234eacaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:51.528776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8de54-735a-419a-af7a-26db234eacaa None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:51.535697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 588/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 12:13:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:53.390753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f290578-7a0b-4c57-835c-6790826b4a40 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:53.391288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f290578-7a0b-4c57-835c-6790826b4a40 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:53.391288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f290578-7a0b-4c57-835c-6790826b4a40 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:53.392634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 579/1167] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:53.398569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e971-1ad0-4017-b0b1-71a3f85048df None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:53.398973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e971-1ad0-4017-b0b1-71a3f85048df None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:53.399140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45e971-1ad0-4017-b0b1-71a3f85048df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:53.406279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:53.417525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-de45e971-1ad0-4017-b0b1-71a3f85048df None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:53.418731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-de45e971-1ad0-4017-b0b1-71a3f85048df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:53.483595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 589/1168] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:53.746187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f290578-7a0b-4c57-835c-6790826b4a40 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:53.841969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f290578-7a0b-4c57-835c-6790826b4a40 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:53.869364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:53.873943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ca658-9bed-4121-98bf-de987a0e15ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:53.877816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ca658-9bed-4121-98bf-de987a0e15ca None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:53.877816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ca658-9bed-4121-98bf-de987a0e15ca None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:53.877816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ca658-9bed-4121-98bf-de987a0e15ca None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:53.877816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ca658-9bed-4121-98bf-de987a0e15ca None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:53.877816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ca658-9bed-4121-98bf-de987a0e15ca None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:53.946852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 580/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Wed Aug 24 12:13:53 2022] POST /identity/v3/domains => generated 267 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:55.903226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad0760-1dc8-420c-ae0a-51523149ff02 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:55.903776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad0760-1dc8-420c-ae0a-51523149ff02 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:55.903776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad0760-1dc8-420c-ae0a-51523149ff02 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:55.905097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 590/1170] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:55.910313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e49ee3-beab-43a8-9837-9cd666529e7e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:55.910736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e49ee3-beab-43a8-9837-9cd666529e7e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:55.910893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e49ee3-beab-43a8-9837-9cd666529e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:55.928773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-56e49ee3-beab-43a8-9837-9cd666529e7e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:55.930519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-56e49ee3-beab-43a8-9837-9cd666529e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:55.993954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 581/1171] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:13:56.260043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad0760-1dc8-420c-ae0a-51523149ff02 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:56.356484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad0760-1dc8-420c-ae0a-51523149ff02 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:56.387970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41acbd7-81a0-446d-ad8c-5ed015f0e4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:56.388569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41acbd7-81a0-446d-ad8c-5ed015f0e4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:56.388848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41acbd7-81a0-446d-ad8c-5ed015f0e4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:56.389116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41acbd7-81a0-446d-ad8c-5ed015f0e4f0 None admin] PATH_INFO: `/v3/domains/f70c57d5ce5f4c74b92a658bdc70ac0c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:56.390945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:56.399695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41acbd7-81a0-446d-ad8c-5ed015f0e4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f70c57d5ce5f4c74b92a658bdc70ac0c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:56.401118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41acbd7-81a0-446d-ad8c-5ed015f0e4f0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:56.402639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 591/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Aug 24 12:13:56 2022] GET /identity/v3/domains/f70c57d5ce5f4c74b92a658bdc70ac0c => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:56.408434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e49ee3-beab-43a8-9837-9cd666529e7e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:56.506113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e49ee3-beab-43a8-9837-9cd666529e7e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:56.527822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:56.539346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e974bbd-5648-4ac2-9235-e821d8a37bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:56.539933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e974bbd-5648-4ac2-9235-e821d8a37bc3 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:56.540208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e974bbd-5648-4ac2-9235-e821d8a37bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:56.540477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e974bbd-5648-4ac2-9235-e821d8a37bc3 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:56.541107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e974bbd-5648-4ac2-9235-e821d8a37bc3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:56.542520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e974bbd-5648-4ac2-9235-e821d8a37bc3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:56.660781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 582/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:13:56 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:13:58.580030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110919d-7d13-4847-813c-5d7f346c95ab None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:58.580604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110919d-7d13-4847-813c-5d7f346c95ab None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:58.580604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110919d-7d13-4847-813c-5d7f346c95ab None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:58.581875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 592/1174] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:13:58.587073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f4b84-5922-4202-b217-1bf213a07291 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:58.587512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f4b84-5922-4202-b217-1bf213a07291 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:58.587621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f4b84-5922-4202-b217-1bf213a07291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:58.605023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0f1f4b84-5922-4202-b217-1bf213a07291 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:13:58.606481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0f1f4b84-5922-4202-b217-1bf213a07291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:13:58.670510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 583/1175] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:13:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:13:58.939648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110919d-7d13-4847-813c-5d7f346c95ab None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:59.038928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110919d-7d13-4847-813c-5d7f346c95ab None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:59.071339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4d84c-1979-4d02-9362-a55a0491727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:59.071941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d84c-1979-4d02-9362-a55a0491727d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:59.072218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d84c-1979-4d02-9362-a55a0491727d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:59.072490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d84c-1979-4d02-9362-a55a0491727d None admin] PATH_INFO: `/v3/domains/f70c57d5ce5f4c74b92a658bdc70ac0c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:59.073579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4d84c-1979-4d02-9362-a55a0491727d None admin] RBAC: Authorizing `identity:get_domain(domain_id=f70c57d5ce5f4c74b92a658bdc70ac0c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:59.075085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4d84c-1979-4d02-9362-a55a0491727d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:59.076610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 593/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Aug 24 12:13:58 2022] GET /identity/v3/domains/f70c57d5ce5f4c74b92a658bdc70ac0c => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:13:59.082136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f4b84-5922-4202-b217-1bf213a07291 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:13:59.175014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f4b84-5922-4202-b217-1bf213a07291 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:13:59.177189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:13:59.207267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f625bb8-3d40-4a40-b8bc-73988d83001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:13:59.207841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f625bb8-3d40-4a40-b8bc-73988d83001b None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:13:59.208118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f625bb8-3d40-4a40-b8bc-73988d83001b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:13:59.208386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f625bb8-3d40-4a40-b8bc-73988d83001b None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:13:59.209033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f625bb8-3d40-4a40-b8bc-73988d83001b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:13:59.211045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f625bb8-3d40-4a40-b8bc-73988d83001b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:13:59.311255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 584/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:13:59 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:14:01.238020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbef4-0029-4503-937f-c7c05d58cddb None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:01.238579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbef4-0029-4503-937f-c7c05d58cddb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:01.238579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbef4-0029-4503-937f-c7c05d58cddb None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:01.239919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 594/1178] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:14:01.245275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90e1e6-d100-4f1a-baa7-968714c691dd None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:01.245650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90e1e6-d100-4f1a-baa7-968714c691dd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:01.245825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90e1e6-d100-4f1a-baa7-968714c691dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:01.264000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3e90e1e6-d100-4f1a-baa7-968714c691dd None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:14:01.265189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3e90e1e6-d100-4f1a-baa7-968714c691dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:14:01.328474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 585/1179] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:14:01.603410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbbef4-0029-4503-937f-c7c05d58cddb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:01.662441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:01.699543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbbef4-0029-4503-937f-c7c05d58cddb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:01.733138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:01.733748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:01.734026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:01.734297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:01.744445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:14:01.749079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:14:01.750256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:14:01.751394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 595/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:14:01.756245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90e1e6-d100-4f1a-baa7-968714c691dd None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:01.841932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:01.850814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90e1e6-d100-4f1a-baa7-968714c691dd None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:01.884637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:01.885167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:01.885447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:01.885717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:01.886372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:14:01.886698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:01.888178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:01.896438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 586/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:01.902762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:01.999873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fc0b9-f896-4d62-94bc-db870c560f68 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:02.021240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:02.032753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:02.033356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:02.033630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:02.033897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] PATH_INFO: `/v3/users/5defc716c9f34b2f90041141e6ae27d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:02.046768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=5defc716c9f34b2f90041141e6ae27d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:02.048236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:02.049741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 596/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:14:01 2022] GET /identity/v3/users/5defc716c9f34b2f90041141e6ae27d1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:02.055766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:02.152557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a1dce-86b0-4d36-898c-d00f3f1f3930 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:02.183512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:02.184088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:02.184362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:02.184627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:02.204782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:02.248311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:02.249718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:02.342775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 587/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:14:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:14:02.348556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:02.445286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f751e36-a329-493d-a858-b857e857c2f4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:02.478616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77790d7f-bf60-40cf-8326-299b2668f31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:02.479258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790d7f-bf60-40cf-8326-299b2668f31d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:02.479535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790d7f-bf60-40cf-8326-299b2668f31d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:02.479831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790d7f-bf60-40cf-8326-299b2668f31d None admin] PATH_INFO: `/v3/projects/b14a0532b2f9482fabccc7d1d0349102` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:02.481178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77790d7f-bf60-40cf-8326-299b2668f31d None admin] RBAC: Authorizing `identity:get_project(project_id=b14a0532b2f9482fabccc7d1d0349102)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:02.482586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77790d7f-bf60-40cf-8326-299b2668f31d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:02.484209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 597/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:14:02 2022] GET /identity/v3/projects/b14a0532b2f9482fabccc7d1d0349102 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:02.490308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:02.541080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:02.586171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe6956-e75d-4cf7-9e61-49fcfdb6382f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:02.619330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9b3c2-e58c-486a-94b3-8dc88b0dae22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:02.619918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9b3c2-e58c-486a-94b3-8dc88b0dae22 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:02.620197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9b3c2-e58c-486a-94b3-8dc88b0dae22 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:02.620471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9b3c2-e58c-486a-94b3-8dc88b0dae22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:02.621187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9b3c2-e58c-486a-94b3-8dc88b0dae22 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=b14a0532b2f9482fabccc7d1d0349102, user.id=5defc716c9f34b2f90041141e6ae27d1) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:14:02.621492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9b3c2-e58c-486a-94b3-8dc88b0dae22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:02.622945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9b3c2-e58c-486a-94b3-8dc88b0dae22 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:02.628157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 588/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:14:02 2022] GET /identity/v3/role_assignments?user.id=5defc716c9f34b2f90041141e6ae27d1&scope.project.id=b14a0532b2f9482fabccc7d1d0349102&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:04.465058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50362-bd37-44a6-b04a-21c136b4f13b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:04.465496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50362-bd37-44a6-b04a-21c136b4f13b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:04.465603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50362-bd37-44a6-b04a-21c136b4f13b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:04.467016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 598/1186] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:14:04.472294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eafe91-16f1-4cf0-a728-bf8c417eee6c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:04.472667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eafe91-16f1-4cf0-a728-bf8c417eee6c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:04.472837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eafe91-16f1-4cf0-a728-bf8c417eee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:04.490595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-95eafe91-16f1-4cf0-a728-bf8c417eee6c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:14:04.491836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-95eafe91-16f1-4cf0-a728-bf8c417eee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:14:04.555800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 589/1187] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:14:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:14:04.822623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50362-bd37-44a6-b04a-21c136b4f13b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:04.872010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:04.921466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50362-bd37-44a6-b04a-21c136b4f13b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:04.954271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:04.954910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:04.955190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:04.955460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:04.965832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:14:04.970487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:14:04.971608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:14:04.972617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 599/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:14:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:14:04.977695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eafe91-16f1-4cf0-a728-bf8c417eee6c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:05.021821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:05.072548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eafe91-16f1-4cf0-a728-bf8c417eee6c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:05.105009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:05.105807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:05.106087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:05.106360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:05.107050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:14:05.107354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:05.108710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:05.116383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 590/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:14:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:05.122028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:05.219109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b7db0-8cab-4e31-ac96-b968a6d1325b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:05.228007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:05.251241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:05.251839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:05.252114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:05.252385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] PATH_INFO: `/v3/users/5defc716c9f34b2f90041141e6ae27d1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:05.253614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] RBAC: Authorizing `identity:get_user(user_id=5defc716c9f34b2f90041141e6ae27d1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:05.255398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:05.257836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 600/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:14:05 2022] GET /identity/v3/users/5defc716c9f34b2f90041141e6ae27d1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:05.263618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:05.358896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9791c72-fa5c-4364-b9c3-24c1a7689239 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:05.367759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:05.391421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41425e-187f-4522-8817-c00aae12386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:05.392030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41425e-187f-4522-8817-c00aae12386c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:05.392306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41425e-187f-4522-8817-c00aae12386c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:05.392576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41425e-187f-4522-8817-c00aae12386c None admin] PATH_INFO: `/v3/projects/b14a0532b2f9482fabccc7d1d0349102` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:05.393666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41425e-187f-4522-8817-c00aae12386c None admin] RBAC: Authorizing `identity:get_project(project_id=b14a0532b2f9482fabccc7d1d0349102)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:05.395120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41425e-187f-4522-8817-c00aae12386c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:05.396937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 591/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:14:05 2022] GET /identity/v3/projects/b14a0532b2f9482fabccc7d1d0349102 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:05.402820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:05.498578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e1a17-c552-4f15-954e-2cae18f1ce22 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:05.532562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffe9f6-7c33-49f2-8778-ec5f18d79ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:05.533198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffe9f6-7c33-49f2-8778-ec5f18d79ae2 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:05.533472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffe9f6-7c33-49f2-8778-ec5f18d79ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:05.533738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffe9f6-7c33-49f2-8778-ec5f18d79ae2 None admin] PATH_INFO: `/v3/projects/b14a0532b2f9482fabccc7d1d0349102/users/5defc716c9f34b2f90041141e6ae27d1/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:05.535627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffe9f6-7c33-49f2-8778-ec5f18d79ae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a0532b2f9482fabccc7d1d0349102, user_id=5defc716c9f34b2f90041141e6ae27d1, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:05.537048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffe9f6-7c33-49f2-8778-ec5f18d79ae2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:05.613756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 601/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:14:05 2022] PUT /identity/v3/projects/b14a0532b2f9482fabccc7d1d0349102/users/5defc716c9f34b2f90041141e6ae27d1/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:14:07.475856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a10e-060c-4919-b356-30a64b844082 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:07.476344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a10e-060c-4919-b356-30a64b844082 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:07.476447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48a10e-060c-4919-b356-30a64b844082 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:07.477743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 592/1193] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:14:07.483215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566c878-66e6-4f3f-8660-160e4e7584c4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:07.483649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566c878-66e6-4f3f-8660-160e4e7584c4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:07.483768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566c878-66e6-4f3f-8660-160e4e7584c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:07.501094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4566c878-66e6-4f3f-8660-160e4e7584c4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:14:07.502267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4566c878-66e6-4f3f-8660-160e4e7584c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:14:07.530515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:07.567180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 602/1194] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:14:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:14:07.829453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48a10e-060c-4919-b356-30a64b844082 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:07.927428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48a10e-060c-4919-b356-30a64b844082 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:07.959410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:07.960148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:07.960425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:07.960691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:07.970643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:14:07.975241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:14:07.976842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:14:07.977694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 593/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:14:07.982991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566c878-66e6-4f3f-8660-160e4e7584c4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:08.077922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566c878-66e6-4f3f-8660-160e4e7584c4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:08.110396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:08.110948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:08.111227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:08.111496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:08.112148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:14:08.112444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:08.113798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:08.121964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 603/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:14:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:08.127555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:08.160714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:08.223345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093f23f-2d28-4d66-a4fb-6c12ee325eba None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:08.256375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:08.257019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:08.257302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:08.257576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] PATH_INFO: `/v3/users/5defc716c9f34b2f90041141e6ae27d1` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:08.258757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] RBAC: Authorizing `identity:get_user(user_id=5defc716c9f34b2f90041141e6ae27d1)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:08.260175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:08.261741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 594/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 12:14:08 2022] GET /identity/v3/users/5defc716c9f34b2f90041141e6ae27d1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:08.267562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:08.295898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:08.364357 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec97e92-6e08-4aa0-91e4-8b45b04f1720 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:08.407661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:08.408514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:08.408925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:08.409322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:08.485702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:08.487170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:08.513035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:08.583145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 604/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:14:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:14:08.589568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:08.684642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e7d51-8b95-4b9a-aa54-3a080370ba4c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:08.686816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:08.718825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441095c-a4c1-4d7b-a211-47e7f91413d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:08.719717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441095c-a4c1-4d7b-a211-47e7f91413d2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:08.720131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441095c-a4c1-4d7b-a211-47e7f91413d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:08.720531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441095c-a4c1-4d7b-a211-47e7f91413d2 None admin] PATH_INFO: `/v3/projects/b14a0532b2f9482fabccc7d1d0349102` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:08.722247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441095c-a4c1-4d7b-a211-47e7f91413d2 None admin] RBAC: Authorizing `identity:get_project(project_id=b14a0532b2f9482fabccc7d1d0349102)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:08.724457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441095c-a4c1-4d7b-a211-47e7f91413d2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:08.726950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 595/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 12:14:08 2022] GET /identity/v3/projects/b14a0532b2f9482fabccc7d1d0349102 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:08.733296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:14:08.829024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121649c7-9564-4ab1-8c82-421e203a71cb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:14:08.855224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:08.862886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bb50a-56f5-4822-aa46-ea17ff54b208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:14:08.871981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bb50a-56f5-4822-aa46-ea17ff54b208 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:14:08.871981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bb50a-56f5-4822-aa46-ea17ff54b208 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:14:08.871981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bb50a-56f5-4822-aa46-ea17ff54b208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:14:08.871981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bb50a-56f5-4822-aa46-ea17ff54b208 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=b14a0532b2f9482fabccc7d1d0349102, user.id=5defc716c9f34b2f90041141e6ae27d1) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:14:08.871981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bb50a-56f5-4822-aa46-ea17ff54b208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:14:08.871981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bb50a-56f5-4822-aa46-ea17ff54b208 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:14:08.874816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 605/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:14:08 2022] GET /identity/v3/role_assignments?user.id=5defc716c9f34b2f90041141e6ae27d1&scope.project.id=b14a0532b2f9482fabccc7d1d0349102&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:14:18.713517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=14 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:14:18.888242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:22.359477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c43211-d165-4559-be40-4d190a9fe1d9 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:22.360059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c43211-d165-4559-be40-4d190a9fe1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:22.360059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c43211-d165-4559-be40-4d190a9fe1d9 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:22.361348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 596/1201] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:22.366395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db984d-1fd6-47da-8625-9782b2207b99 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:22.366864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db984d-1fd6-47da-8625-9782b2207b99 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:22.366963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db984d-1fd6-47da-8625-9782b2207b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:22.384906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-21db984d-1fd6-47da-8625-9782b2207b99 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:22.386069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-21db984d-1fd6-47da-8625-9782b2207b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:22.456249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 606/1202] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:22.723734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c43211-d165-4559-be40-4d190a9fe1d9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:22.824668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c43211-d165-4559-be40-4d190a9fe1d9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:22.858349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:22.859009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:22.859291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:22.859564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:22.870148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:22.871613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:22.886088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:22.887321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:22.888532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:22.888532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:22.888532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:22.888532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 597/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:22.893515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db984d-1fd6-47da-8625-9782b2207b99 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:22.990289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db984d-1fd6-47da-8625-9782b2207b99 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:23.019173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:23.024832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c27bd-f8b3-4b6b-b969-b421a1ba19d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:23.025422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c27bd-f8b3-4b6b-b969-b421a1ba19d2 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:23.025698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c27bd-f8b3-4b6b-b969-b421a1ba19d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:23.025967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c27bd-f8b3-4b6b-b969-b421a1ba19d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:23.026632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c27bd-f8b3-4b6b-b969-b421a1ba19d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:23.026975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c27bd-f8b3-4b6b-b969-b421a1ba19d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:23.028385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c27bd-f8b3-4b6b-b969-b421a1ba19d2 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:23.039175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 607/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:23.045770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:23.097576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:23.144873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08127e-2a3c-4407-9c2f-cc5e6c1b617f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:23.178481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef70f0-20db-4929-b030-aacdbb4dea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:23.179104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef70f0-20db-4929-b030-aacdbb4dea22 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:23.179380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef70f0-20db-4929-b030-aacdbb4dea22 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:23.179709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef70f0-20db-4929-b030-aacdbb4dea22 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:23.180332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef70f0-20db-4929-b030-aacdbb4dea22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:23.182147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef70f0-20db-4929-b030-aacdbb4dea22 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:23.280320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 598/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 12:15:23 2022] POST /identity/v3/users => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:15:25.206639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27cb45-b52c-4720-9608-ead20238170f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:25.207199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27cb45-b52c-4720-9608-ead20238170f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:25.207199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27cb45-b52c-4720-9608-ead20238170f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:25.208680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 608/1206] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:15:25.213860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d06f8-ec2b-49c7-9a5b-8c83bba22c77 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:25.214307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d06f8-ec2b-49c7-9a5b-8c83bba22c77 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:25.214410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d06f8-ec2b-49c7-9a5b-8c83bba22c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:25.234738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-528d06f8-ec2b-49c7-9a5b-8c83bba22c77 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:25.239944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-528d06f8-ec2b-49c7-9a5b-8c83bba22c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:25.302897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 599/1207] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:25.570011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27cb45-b52c-4720-9608-ead20238170f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:25.669589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27cb45-b52c-4720-9608-ead20238170f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:25.702603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:25.703236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:25.703511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:25.703774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:25.713491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:15:25.718148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:25.723795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:25.724760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 609/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:25.725603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d06f8-ec2b-49c7-9a5b-8c83bba22c77 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:25.764231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:25.821472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d06f8-ec2b-49c7-9a5b-8c83bba22c77 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:25.854572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:25.855209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:25.855489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:25.855763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:25.856966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:25.857271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:25.858632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:25.868149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 600/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:25.874873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:25.925565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:25.973239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1a1a9-78cb-464d-be5f-714563ef4ec0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:26.006848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:26.007455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:26.007730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:26.007996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:26.017457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:26.018906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:26.032675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:26.033801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:26.034797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:26.034797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:26.034797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:26.034797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 610/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:26.039969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:26.133765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:26.136589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e5fbc-7698-4817-9a6c-23d7b31d9fc2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:26.168421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:26.169583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:26.169862 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:26.170161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:26.170823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:26.171124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:26.172476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:26.183551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 601/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:26.189296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:26.286061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364012-cf7e-4787-a26f-4d2a8598d3bb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:26.288621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:26.326376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:26.327015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:26.327292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:26.327564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:26.334209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:26.335665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:15:26.347194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:26.348298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:26.349715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 611/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:26.354074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:26.451926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972fbb1-5470-4b17-9ca5-9463711cc13e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:26.484742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:26.485333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:26.485608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:26.485875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:26.486595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:26.486923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:26.488336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:26.505579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 602/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 12:15:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:26.511567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:26.614873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0047a-7945-4fbb-bc03-c7c64fd16394 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:26.648123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:26.648714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:26.648989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:26.649256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:26.679681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:26.717081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:26.718514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:26.813568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 612/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:15:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:15:26.819664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:26.838785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:26.916984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a3872-ec03-4046-9270-2a65075f038d None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:26.953978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:26.954531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:26.954838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:26.955110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:26.965868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:26.967483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:26.983222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:26.984342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:26.985457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:26.985457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:26.985457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:26.985457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 603/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:26.990086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:27.050213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:27.088169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82cc47-abc5-4ec2-87a4-caaf105ed7ee None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:27.120224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:27.120809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:27.121081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:27.121347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:27.122471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:27.122794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:27.124145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:27.135927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 613/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:27.143211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:27.211567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:27.243397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876e474-b4b9-4e0a-b4e7-c34c9065e47e None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:27.278183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:27.278926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:27.279200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:27.279462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:27.285929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:27.287378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:15:27.298679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:27.299723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:27.301077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 604/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:27.305644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:27.401909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dba50b-f60e-4037-a4a0-b1b2e12bd1de None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:27.425192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:27.437755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d31f0-7034-4fcc-b35e-6438b494e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:27.438356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d31f0-7034-4fcc-b35e-6438b494e2cc None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:27.438633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d31f0-7034-4fcc-b35e-6438b494e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:27.438930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d31f0-7034-4fcc-b35e-6438b494e2cc None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:27.439611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d31f0-7034-4fcc-b35e-6438b494e2cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:27.439908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d31f0-7034-4fcc-b35e-6438b494e2cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:27.441327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d31f0-7034-4fcc-b35e-6438b494e2cc None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:27.452708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 614/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:27.458822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:27.555215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70252-0ec6-4b75-8ce5-2af533cc5503 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:27.583916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:27.589871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f732276-a729-4713-92a9-00ca5742bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:27.590461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f732276-a729-4713-92a9-00ca5742bec4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:27.591289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f732276-a729-4713-92a9-00ca5742bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:27.591559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f732276-a729-4713-92a9-00ca5742bec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:27.592277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f732276-a729-4713-92a9-00ca5742bec4 None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=258097db9804464c81d52fb4aada32ee) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:27.592577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f732276-a729-4713-92a9-00ca5742bec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:27.594050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f732276-a729-4713-92a9-00ca5742bec4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:27.599254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 605/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:15:27 2022] GET /identity/v3/role_assignments?user.id=258097db9804464c81d52fb4aada32ee&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:29.446446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31307142-6def-4e96-a9c0-430ad6003181 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:29.446974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31307142-6def-4e96-a9c0-430ad6003181 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:29.447077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31307142-6def-4e96-a9c0-430ad6003181 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:29.448401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 615/1220] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:29.454207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0c419-a395-41ef-9b54-b8aadefac72f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:29.454597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0c419-a395-41ef-9b54-b8aadefac72f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:29.454775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0c419-a395-41ef-9b54-b8aadefac72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:29.472479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-66f0c419-a395-41ef-9b54-b8aadefac72f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:29.473665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-66f0c419-a395-41ef-9b54-b8aadefac72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:29.540456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 606/1221] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:29.808542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31307142-6def-4e96-a9c0-430ad6003181 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:29.912678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31307142-6def-4e96-a9c0-430ad6003181 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:29.945360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:29.945964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:29.946243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:29.946567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:29.956315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:15:29.960888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:29.962801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:29.964060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 616/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:29.968542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0c419-a395-41ef-9b54-b8aadefac72f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:30.064711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0c419-a395-41ef-9b54-b8aadefac72f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:30.089952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:30.100974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:30.101567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:30.101844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:30.102109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:30.102805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] RBAC: Adding query filter params (name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:30.106863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:30.106863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:30.113021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 607/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:30.118763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:30.161612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:30.216066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23483bf-ec60-4c73-a972-8a31080d0450 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:30.249749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:30.250388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:30.250890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:30.251323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:30.261440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:30.262915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:30.276716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:30.277852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:30.279012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:30.279012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:30.279012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:30.279012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 617/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:30.283681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:30.382780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a0f9-bb1d-4ee0-9bf1-2f5e2bdf4e84 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:30.414476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:30.415100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:30.415373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:30.415654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:30.416271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:30.416567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:30.417914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:30.431389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 608/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:15:30.437676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:30.532006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:30.539086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6432a-7b6e-418d-80e9-a2c2676a1d1e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:30.572868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:30.573475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:30.573746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:30.574011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:30.580993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:30.582399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:15:30.593990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:30.595127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:30.596190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 618/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:30.600980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:30.647231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:30.697613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4747d9a-1e5b-489b-92d7-e77096488aa4 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:30.736424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:30.737024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:30.737307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:30.737575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:30.738294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:30.738593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:30.740032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:30.756370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 609/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 12:15:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:30.762128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:30.860917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6487e54-f85e-420e-b61a-99c744a5a963 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:30.894730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:30.895387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:30.895665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:30.895934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:30.913473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:30.913473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:30.922086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:30.923100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:30.924026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:30.924026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:30.924026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:30.924026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 619/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:30.929107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:31.026172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036326b-70fe-4804-b814-95d4f2baa5c9 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:31.029082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:31.059496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:31.060090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:31.060366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:31.060635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:31.061328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:31.061628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:31.063117 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:31.073774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 610/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:31.080173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:31.089379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:31.179115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379765d-09ce-4a84-8813-023a89bfc105 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:31.211701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:31.212315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:31.212595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:31.212865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:31.219807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:31.221228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:15:31.232287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:31.233387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:31.234709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 620/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:31.239247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:31.335502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522fafe-cc53-42c1-a52a-ba6ed19fe0b2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:31.368388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38174682-bd3a-46e2-b8ac-b78f1aad8633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:31.368974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174682-bd3a-46e2-b8ac-b78f1aad8633 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:31.369251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174682-bd3a-46e2-b8ac-b78f1aad8633 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:31.369523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174682-bd3a-46e2-b8ac-b78f1aad8633 None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:31.370327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38174682-bd3a-46e2-b8ac-b78f1aad8633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:31.370631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38174682-bd3a-46e2-b8ac-b78f1aad8633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:31.372711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38174682-bd3a-46e2-b8ac-b78f1aad8633 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:31.384432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 611/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:31.391131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:31.455748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:31.488686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b7db7-67f7-4402-97de-54d02f86a7c3 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:31.521282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708996b6-6c47-442f-8e16-5fe26c9ce516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:31.521918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708996b6-6c47-442f-8e16-5fe26c9ce516 None admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:31.522197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708996b6-6c47-442f-8e16-5fe26c9ce516 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:31.522468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708996b6-6c47-442f-8e16-5fe26c9ce516 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/258097db9804464c81d52fb4aada32ee/roles/c26057e0945745a39399a115b756ea2b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:31.537659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708996b6-6c47-442f-8e16-5fe26c9ce516 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=258097db9804464c81d52fb4aada32ee, role_id=c26057e0945745a39399a115b756ea2b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:31.539113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708996b6-6c47-442f-8e16-5fe26c9ce516 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:31.629660 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 621/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:15:31 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/258097db9804464c81d52fb4aada32ee/roles/c26057e0945745a39399a115b756ea2b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:15:33.513194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9058151-619a-4193-9455-2dce6cf97dd5 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:33.513745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9058151-619a-4193-9455-2dce6cf97dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:33.513745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9058151-619a-4193-9455-2dce6cf97dd5 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:33.515123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 612/1233] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:33.520289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd199b-019d-4da2-8bcc-7bef69ea58b5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:33.521911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd199b-019d-4da2-8bcc-7bef69ea58b5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:33.521911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd199b-019d-4da2-8bcc-7bef69ea58b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:33.538568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-07fd199b-019d-4da2-8bcc-7bef69ea58b5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:33.539812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-07fd199b-019d-4da2-8bcc-7bef69ea58b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:33.604067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 622/1234] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:33.870958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9058151-619a-4193-9455-2dce6cf97dd5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:33.895729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:33.969862 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9058151-619a-4193-9455-2dce6cf97dd5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:34.003811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:34.004432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:34.004714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:34.004987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:34.015274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:15:34.019914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:34.021032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:34.022104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 613/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 12:15:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:34.027234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd199b-019d-4da2-8bcc-7bef69ea58b5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:34.072403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:34.124665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd199b-019d-4da2-8bcc-7bef69ea58b5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:34.158633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:34.159269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:34.159543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:34.159809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:34.160565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] RBAC: Adding query filter params (name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:34.160861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:34.162215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:34.170015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 623/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 12:15:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:34.175742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:34.257764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:34.273189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235b3a0-33aa-41e5-929f-1a7a797b3fcf None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:34.305500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:34.306113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:34.306387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:34.306674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:34.316780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:34.318225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:34.334857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:34.336011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:34.336899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:34.336899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:34.336899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:34.336899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 614/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:34.341888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:34.448239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107749f-6e92-43fa-950a-dd1b0872350c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:34.457324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:34.480774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:34.481360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:34.481634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:34.481898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:34.482520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:34.482843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:34.484209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:34.495269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 624/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:34.501780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:34.599171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26374acb-a049-4ccc-b57b-087acb1fa25f None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:34.632636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:34.633250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:34.633523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:34.633788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:34.636359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:34.641712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:34.643169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:15:34.654091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:34.655213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:34.656399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 615/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:34.660993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:34.759707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7fb5-6ab4-49a6-b0f3-4419db2ed249 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:34.791656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:34.792237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:34.792511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:34.792777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:34.793986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:34.794281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:34.795716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:34.812541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 625/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 12:15:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:34.818444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:34.915181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fd085-c94f-4f6b-86b0-60e06b79896c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:34.948356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:34.948949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:34.949228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:34.949499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:35.009101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:35.015511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:35.016902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:35.110394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 616/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:15:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:15:35.116325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:35.142707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:35.212209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced5e84-5ea9-4b76-9b63-0d7c16b32f03 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:35.247249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:35.247861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:35.248134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:35.248403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:35.257991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:35.261240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:35.273522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:35.274696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:35.275848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:35.275848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:35.275848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:35.275848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 626/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:35.280964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:35.372979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:35.378629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70690456-68e9-433d-b9a0-c6d86218bb45 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:35.412192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:35.412882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:35.413145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:35.413419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:35.414081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:35.414380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:35.415837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:35.427893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 617/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:35.433647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:35.509877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:35.531081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62770d65-c423-4f0c-b26a-12543866942c None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:35.565416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:35.566032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:35.566306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:35.566576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:35.573173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:35.575103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:15:35.585938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:35.586990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:35.588289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 627/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:35.593017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:35.691720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fbf73-3b89-4b54-9ca4-9ebf5da86838 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:35.723318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951bf6a-9e97-4e16-89ef-46b6271ba56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:35.723940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951bf6a-9e97-4e16-89ef-46b6271ba56b None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:35.724219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951bf6a-9e97-4e16-89ef-46b6271ba56b None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:35.724489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951bf6a-9e97-4e16-89ef-46b6271ba56b None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:35.725167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951bf6a-9e97-4e16-89ef-46b6271ba56b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:35.725466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951bf6a-9e97-4e16-89ef-46b6271ba56b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:35.726913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951bf6a-9e97-4e16-89ef-46b6271ba56b None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:35.739308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 618/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:35.745554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:35.844775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32adbce-545c-4290-b45d-c35234ddda94 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:35.879142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58dc08-98ac-4360-b391-bce427f3120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:35.879726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc08-98ac-4360-b391-bce427f3120e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:35.880004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc08-98ac-4360-b391-bce427f3120e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:35.880275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dc08-98ac-4360-b391-bce427f3120e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:35.880993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58dc08-98ac-4360-b391-bce427f3120e None admin] RBAC: Adding query filter params (role.id=c26057e0945745a39399a115b756ea2b, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=258097db9804464c81d52fb4aada32ee) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:35.881299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58dc08-98ac-4360-b391-bce427f3120e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:35.882743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58dc08-98ac-4360-b391-bce427f3120e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:35.888407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 628/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:15:35 2022] GET /identity/v3/role_assignments?user.id=258097db9804464c81d52fb4aada32ee&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=c26057e0945745a39399a115b756ea2b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:37.802870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63b3c-faaf-4145-a990-bd29dc607077 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:37.803428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63b3c-faaf-4145-a990-bd29dc607077 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:37.803428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63b3c-faaf-4145-a990-bd29dc607077 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:37.804765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 619/1247] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:37.809857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783648ff-06c0-4cb8-898e-af50541c5d16 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:37.810242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783648ff-06c0-4cb8-898e-af50541c5d16 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:37.810399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783648ff-06c0-4cb8-898e-af50541c5d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:37.816595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:37.828730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-783648ff-06c0-4cb8-898e-af50541c5d16 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:37.829903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-783648ff-06c0-4cb8-898e-af50541c5d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:37.892876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 629/1248] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:38.156682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c63b3c-faaf-4145-a990-bd29dc607077 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:38.175209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:38.263458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c63b3c-faaf-4145-a990-bd29dc607077 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:38.296402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:38.297010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:38.297284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:38.297589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:38.307128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:15:38.312227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:38.313360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:38.314258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 620/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:38.319374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783648ff-06c0-4cb8-898e-af50541c5d16 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:38.418103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783648ff-06c0-4cb8-898e-af50541c5d16 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:38.447133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:38.452645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:38.453233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:38.453508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:38.453774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:38.454425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:38.454748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:38.456099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:38.463875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 630/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:15:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:38.469746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:38.550790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:38.573162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d5067-0b73-49ec-9c9a-0af667ba4415 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:38.605220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:38.609501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:38.609501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:38.609501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:38.616370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:38.617854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:38.632102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:38.633151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:38.634252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:38.634252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:38.634252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:38.634252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 621/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:38.639422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:38.737018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449f47d-9df1-41a3-b8c1-43d7be546176 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:38.770083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:38.770689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:38.770966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:38.775661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:38.775661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:38.775661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:38.775661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:38.786366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 631/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:38.792331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:38.891869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e357a1-bd44-423b-9558-b2623fef8a96 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:38.917073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:38.924942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:38.925559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:38.925855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:38.926124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:38.934665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:38.936099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:15:38.946729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:38.947836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:38.948805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 622/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:38.954389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:39.008128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:39.050038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83c57f-f949-4aae-9fe8-ea2b0289fee0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:39.083366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:39.083988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:39.084261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:39.084527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:39.085232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:39.085530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:39.086969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:39.103617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 632/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 12:15:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:39.109604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:39.205865 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67ea2e-74f2-4efe-9f7f-673ecaddb286 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:39.238333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:39.238951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:39.239228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:39.239492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:39.292519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:39.304818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:39.306258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:39.401001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 623/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:15:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:15:39.406843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:39.503649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e231601-2dab-4d89-bb1c-f03a0fcddcb5 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:39.516378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:39.536146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:39.536751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:39.537025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:39.537292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:39.547230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:39.548589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:39.563188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:39.564312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:39.565271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:39.565271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:39.565271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:39.565271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 633/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:39.569998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:39.654483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:39.666010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9c927-c06d-4f1e-8b68-1f6c45f78e45 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:39.698333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:39.698968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:39.699244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:39.704274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:39.704274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:39.704274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:39.704274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:39.712582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 624/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:39.718920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:39.815578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a5f76-c8e9-4539-a45e-9f310799a875 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:39.848076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:39.848684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:39.848958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:39.849225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:39.856625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:39.858037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:15:39.868873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:39.869959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:39.870959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 634/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:39.876022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:39.973305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee060d-56ce-4a1c-a0ee-d614f56c0575 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:40.005635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056627bd-f4e7-48f9-9e61-9a029158522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:40.006223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056627bd-f4e7-48f9-9e61-9a029158522c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:40.006500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056627bd-f4e7-48f9-9e61-9a029158522c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:40.006799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056627bd-f4e7-48f9-9e61-9a029158522c None admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:40.007486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056627bd-f4e7-48f9-9e61-9a029158522c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:40.007788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056627bd-f4e7-48f9-9e61-9a029158522c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:40.009211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056627bd-f4e7-48f9-9e61-9a029158522c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:40.021296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 625/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:40.027416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:40.052897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:40.124504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e2dea-e667-4747-9c82-a58464c876c4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:40.157386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44fc1-5640-48ab-9140-46eb9aeb9957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:40.158048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44fc1-5640-48ab-9140-46eb9aeb9957 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:40.158323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44fc1-5640-48ab-9140-46eb9aeb9957 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:40.158587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44fc1-5640-48ab-9140-46eb9aeb9957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:40.159321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44fc1-5640-48ab-9140-46eb9aeb9957 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=258097db9804464c81d52fb4aada32ee) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:40.159624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44fc1-5640-48ab-9140-46eb9aeb9957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:40.161030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44fc1-5640-48ab-9140-46eb9aeb9957 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:40.166140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 635/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:15:40 2022] GET /identity/v3/role_assignments?user.id=258097db9804464c81d52fb4aada32ee&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=753fd078fdac40abb65aa43057d79c62 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:42.047621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b10b20-0286-4866-91c6-defc9053fb4f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:42.048174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b10b20-0286-4866-91c6-defc9053fb4f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:42.048174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b10b20-0286-4866-91c6-defc9053fb4f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:42.049493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 626/1261] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:42.054752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89f136-2e3c-45fc-bcd3-b2f0605dc29d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:42.055120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89f136-2e3c-45fc-bcd3-b2f0605dc29d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:42.055383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89f136-2e3c-45fc-bcd3-b2f0605dc29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:42.073074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2a89f136-2e3c-45fc-bcd3-b2f0605dc29d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:42.074243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2a89f136-2e3c-45fc-bcd3-b2f0605dc29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:42.139677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 636/1262] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:42.407128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b10b20-0286-4866-91c6-defc9053fb4f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:42.417964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:42.507847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b10b20-0286-4866-91c6-defc9053fb4f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:42.542086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:42.542723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:42.543000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:42.543312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:42.553039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:15:42.557762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:42.559319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:42.560221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 627/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:15:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:42.565445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89f136-2e3c-45fc-bcd3-b2f0605dc29d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:42.583762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:42.662936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89f136-2e3c-45fc-bcd3-b2f0605dc29d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:42.697353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:42.697942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:42.698214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:42.698477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:42.699161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:42.700180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:42.701532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:42.709345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 637/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:42.715185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:42.780408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:42.812888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e9b92-4fa0-43e6-84f2-a5c4060d0d15 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:42.845266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:42.845887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:42.846164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:42.846429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:42.857172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:42.858601 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:42.872960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:42.874060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:42.875011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:42.875011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:42.875011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:42.875011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 628/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:42.879879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:42.959638 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:42.978328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a07855-8709-41b0-bb3c-e320fd309819 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:43.013592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:43.014186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:43.014463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:43.014758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:43.015426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:43.015723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:43.017073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:43.028573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 638/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:43.034166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:43.131529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5230f8-bb90-4c25-8c11-df2a7d4b0ae7 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:43.151160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:43.165165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:43.165784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:43.166061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:43.166329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:43.173487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:43.174970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:15:43.185572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:43.186686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:43.187637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 629/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:43.192451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:43.290359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613fd0a-f7df-470b-9ca7-0918f0004e73 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:43.325701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:43.326293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:43.326568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:43.326870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:43.327589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:43.327892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:43.329311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:43.338558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:43.346452 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 639/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 12:15:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:43.352178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:43.451286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6b001-7a5b-4d36-966c-9b44344fa82c None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:43.483965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:43.484578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:43.484856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:43.485127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:43.495256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:43.497240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:43.510845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:43.513921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:43.515046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:43.515046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:43.515046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:43.515046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 630/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:43.518490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:43.615152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc71713-2a37-4c2c-ab99-5e17dffd647b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:43.648542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:43.649131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:43.649412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:43.650458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:43.650458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:43.650739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:43.651932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:43.662765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 640/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:43.668401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:43.699923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:43.766185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fdfaa-fec2-4160-ab9a-3c7f6269fc21 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:43.799037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:43.799646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:43.799921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:43.800185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:43.806439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:43.807901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:15:43.819891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:43.821058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:43.822331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 631/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:43.826917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:43.883577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:43.923962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34d3b3-b31e-414f-9c29-553722443ea6 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:43.957215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecd363-1008-4059-aa95-ad7ff70cf95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:43.957805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd363-1008-4059-aa95-ad7ff70cf95a None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:43.958082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd363-1008-4059-aa95-ad7ff70cf95a None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:43.958352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecd363-1008-4059-aa95-ad7ff70cf95a None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:43.959065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecd363-1008-4059-aa95-ad7ff70cf95a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:43.959370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecd363-1008-4059-aa95-ad7ff70cf95a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:43.960921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecd363-1008-4059-aa95-ad7ff70cf95a None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:43.972672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 641/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:43.978935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:44.070529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:44.078858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cc1b9-67e3-4231-85ea-e4a585c1300a None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:44.112112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5fe11-8184-47aa-ade7-5bff8aad01a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:44.112784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5fe11-8184-47aa-ade7-5bff8aad01a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:44.113061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5fe11-8184-47aa-ade7-5bff8aad01a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:44.113329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5fe11-8184-47aa-ade7-5bff8aad01a1 None admin] PATH_INFO: `/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/258097db9804464c81d52fb4aada32ee/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:44.115246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5fe11-8184-47aa-ade7-5bff8aad01a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aff3a8a82b04c4798539c93ca7ee2b4, user_id=258097db9804464c81d52fb4aada32ee, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:44.116685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5fe11-8184-47aa-ade7-5bff8aad01a1 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:44.216820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 632/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 12:15:43 2022] PUT /identity/v3/projects/1aff3a8a82b04c4798539c93ca7ee2b4/users/258097db9804464c81d52fb4aada32ee/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:15:46.091755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2191b-8771-45c1-953c-c3413d57ae5f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:46.092281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2191b-8771-45c1-953c-c3413d57ae5f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:46.092281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2191b-8771-45c1-953c-c3413d57ae5f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:46.093622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 642/1274] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:46.098924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400406fc-0f37-4c69-95c1-8bb784c75753 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:46.099361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400406fc-0f37-4c69-95c1-8bb784c75753 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:46.099459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400406fc-0f37-4c69-95c1-8bb784c75753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:46.117531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-400406fc-0f37-4c69-95c1-8bb784c75753 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:46.118738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-400406fc-0f37-4c69-95c1-8bb784c75753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:46.184246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 633/1275] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:15:46.456473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2191b-8771-45c1-953c-c3413d57ae5f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:46.556055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2191b-8771-45c1-953c-c3413d57ae5f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:46.572893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:46.588823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:46.589433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:46.590210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:46.590485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:46.600226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:15:46.605062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:46.606768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:46.607825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 643/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 12:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:46.612641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400406fc-0f37-4c69-95c1-8bb784c75753 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:46.691420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:46.708731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400406fc-0f37-4c69-95c1-8bb784c75753 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:46.741384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:46.741974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:46.742251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:46.742519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:46.743202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:46.743501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:46.744920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:46.752603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 634/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 12:15:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:46.758306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:46.859241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7102d4-e8bc-40db-9ce3-06ea481e44e0 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:46.892028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:46.892641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:46.892940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:46.893211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:46.903495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:46.904925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:46.918792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:46.919904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:46.920854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:46.920854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:46.920854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:46.920854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 644/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:46.925900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:47.024103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2c82a-560e-4ed7-b72a-4accddbe5fb2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:47.056891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:47.057482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:47.057764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:47.058036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:47.059231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:47.059532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:47.060963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:47.072573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 635/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:47.078918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:47.108824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:47.175763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ada99-3101-4c84-bf4e-53e7c6d34a42 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:47.210278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:47.210918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:47.211192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:47.211461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:47.218233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:47.219679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:15:47.230983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:47.232161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:47.233840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 645/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:47.237945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:47.241764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:47.335648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a191604-0e30-4171-9ab2-32a5f186a422 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:47.369109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:47.369696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:47.369971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:47.370236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:47.371016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:47.371316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:47.372744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:47.389670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 636/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 12:15:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:47.395526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:47.483202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:47.492224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0661fa-165a-4e22-bb41-6dd2c6e008e4 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:47.524078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:47.524657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:47.524932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:47.525196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:47.592192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:47.593567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:47.678460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:47.687981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 646/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 12:15:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:15:47.693777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:47.754404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:47.791447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f831f-e6ea-477b-b9a8-38481bf2bbc2 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:47.824949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:47.825564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:47.825837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:47.826102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:47.835676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:47.837107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:15:47.851474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:47.852551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:15:47.853661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:15:47.853661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:15:47.853661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:47.853661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 637/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:47.859099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:47.954636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62200b-a2ab-4a4f-b39d-c31972439eca None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:47.987861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:47.988445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:47.988721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:47.988989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:47.989615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:47.989913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:47.991293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:48.002350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 647/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 12:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:48.008223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:48.106430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aaa37c-be44-4328-aab7-5d26fec38137 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:48.119890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:48.139049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:48.141064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:48.141064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:48.141064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:48.146719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:48.148153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:15:48.159577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:48.160664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: ERROR keystone.server.flask.application Aug 24 12:15:48.161827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 638/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 12:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:48.166561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:48.215559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:48.264245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b5ced-5223-4435-8d5b-b5912731d38b None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:48.296689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d15e8-cb5f-4b3b-bd8b-e09254d82bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:48.297276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d15e8-cb5f-4b3b-bd8b-e09254d82bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:48.297551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d15e8-cb5f-4b3b-bd8b-e09254d82bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:48.297818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d15e8-cb5f-4b3b-bd8b-e09254d82bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:48.298499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d15e8-cb5f-4b3b-bd8b-e09254d82bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:48.298838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d15e8-cb5f-4b3b-bd8b-e09254d82bc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:48.300260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d15e8-cb5f-4b3b-bd8b-e09254d82bc0 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:48.311756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 648/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 12:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:48.317854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:48.414979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5669a4-8be1-4342-817c-b54b89cf44df None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:48.447868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a84777-33c9-4836-af2c-e37274b2e4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:48.448464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84777-33c9-4836-af2c-e37274b2e4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:48.448742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84777-33c9-4836-af2c-e37274b2e4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:48.449530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84777-33c9-4836-af2c-e37274b2e4d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:48.450244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a84777-33c9-4836-af2c-e37274b2e4d3 None admin] RBAC: Adding query filter params (role.id=753fd078fdac40abb65aa43057d79c62, scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4, user.id=258097db9804464c81d52fb4aada32ee) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:48.450550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a84777-33c9-4836-af2c-e37274b2e4d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:48.452006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a84777-33c9-4836-af2c-e37274b2e4d3 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:48.457577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 639/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 12:15:48 2022] GET /identity/v3/role_assignments?user.id=258097db9804464c81d52fb4aada32ee&scope.project.id=1aff3a8a82b04c4798539c93ca7ee2b4&role.id=753fd078fdac40abb65aa43057d79c62 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:50.375625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635834f6-e352-463b-a5d9-e424d497c424 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:50.376300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635834f6-e352-463b-a5d9-e424d497c424 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:50.376300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635834f6-e352-463b-a5d9-e424d497c424 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:50.377523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 649/1288] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:50.382821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f144cd-e54a-4c38-929e-95a110e7138e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:50.383241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f144cd-e54a-4c38-929e-95a110e7138e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:50.383362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f144cd-e54a-4c38-929e-95a110e7138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:50.401376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e7f144cd-e54a-4c38-929e-95a110e7138e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:50.404661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e7f144cd-e54a-4c38-929e-95a110e7138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:50.435101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:50.473451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 640/1289] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:50.738878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635834f6-e352-463b-a5d9-e424d497c424 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:50.846619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635834f6-e352-463b-a5d9-e424d497c424 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:50.858914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:50.884179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:50.884816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:50.885088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:50.885353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] PATH_INFO: `/v3/services/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:50.885935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:50.887914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:50.896978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:50.898347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:50.899311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:50.902846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 650/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:15:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:50.904464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f144cd-e54a-4c38-929e-95a110e7138e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:51.009803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f144cd-e54a-4c38-929e-95a110e7138e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:51.045812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51157b-b508-4329-b8b6-3a20b64a7373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:51.046414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51157b-b508-4329-b8b6-3a20b64a7373 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:51.046724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51157b-b508-4329-b8b6-3a20b64a7373 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:51.047000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51157b-b508-4329-b8b6-3a20b64a7373 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:51.047630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51157b-b508-4329-b8b6-3a20b64a7373 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:51.047933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51157b-b508-4329-b8b6-3a20b64a7373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:51.049292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51157b-b508-4329-b8b6-3a20b64a7373 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:51.054133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 641/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 12:15:50 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:51.061029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:51.162690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ad0d5-d519-4c27-bc33-29a3746f3cdb None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:51.195885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c487a7-0dc5-4804-996e-2dfa6bf4b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:51.196514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c487a7-0dc5-4804-996e-2dfa6bf4b32d None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:51.196789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c487a7-0dc5-4804-996e-2dfa6bf4b32d None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:51.197059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c487a7-0dc5-4804-996e-2dfa6bf4b32d None admin] PATH_INFO: `/v3/services` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:51.197805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c487a7-0dc5-4804-996e-2dfa6bf4b32d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:51.198115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c487a7-0dc5-4804-996e-2dfa6bf4b32d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:51.200090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c487a7-0dc5-4804-996e-2dfa6bf4b32d None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:51.204501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 651/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 12:15:51 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:15:53.079104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d44795-9e3f-4b06-95fe-712b7cb6f20f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:53.079639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d44795-9e3f-4b06-95fe-712b7cb6f20f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:53.079639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d44795-9e3f-4b06-95fe-712b7cb6f20f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:53.080999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 642/1293] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:53.086218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdded52-a68b-4166-9d4f-e8663d69be3c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:53.086637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdded52-a68b-4166-9d4f-e8663d69be3c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:53.086771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdded52-a68b-4166-9d4f-e8663d69be3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:53.106324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2bdded52-a68b-4166-9d4f-e8663d69be3c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:53.107564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2bdded52-a68b-4166-9d4f-e8663d69be3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:53.136148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:53.172549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 652/1294] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:53.437574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d44795-9e3f-4b06-95fe-712b7cb6f20f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:53.469348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:53.537440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d44795-9e3f-4b06-95fe-712b7cb6f20f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:53.569332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fee927-2e6b-484a-b491-311e09ddec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:53.569912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fee927-2e6b-484a-b491-311e09ddec30 None admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:53.570187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fee927-2e6b-484a-b491-311e09ddec30 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:53.570455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fee927-2e6b-484a-b491-311e09ddec30 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:53.571120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fee927-2e6b-484a-b491-311e09ddec30 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:53.572471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fee927-2e6b-484a-b491-311e09ddec30 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:53.656048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 643/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 12:15:53 2022] POST /identity/v3/services => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:15:55.565228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a3e92-3f88-4a10-894d-da1608466558 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:55.565770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a3e92-3f88-4a10-894d-da1608466558 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:55.565770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a3e92-3f88-4a10-894d-da1608466558 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:55.567148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 653/1296] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:55.572482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff20f6-96e0-4145-948d-9849cb974224 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:55.572870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff20f6-96e0-4145-948d-9849cb974224 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:55.573034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff20f6-96e0-4145-948d-9849cb974224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:55.591072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a1ff20f6-96e0-4145-948d-9849cb974224 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:55.592300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a1ff20f6-96e0-4145-948d-9849cb974224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:55.663600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 644/1297] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:55 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:55.933136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a3e92-3f88-4a10-894d-da1608466558 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:56.033090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a3e92-3f88-4a10-894d-da1608466558 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:56.066498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:56.067129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:56.067404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:56.067669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:56.068236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:56.069583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:56.078803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:56.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:56.081088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 654/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:15:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:56.085909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff20f6-96e0-4145-948d-9849cb974224 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:56.104135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:56.180168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff20f6-96e0-4145-948d-9849cb974224 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:56.214081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:56.214638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:56.214942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:56.215230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:56.215958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:56.216256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:56.217606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:56.222575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 645/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 12:15:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:56.228583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:56.259705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:56.326758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db803f7-0f02-4b03-9a99-aeeb0e687144 None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:56.360411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c437b-0f71-4e14-bd65-6f8a0472775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:56.360999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c437b-0f71-4e14-bd65-6f8a0472775e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:56.361276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c437b-0f71-4e14-bd65-6f8a0472775e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:56.361545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c437b-0f71-4e14-bd65-6f8a0472775e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:56.362243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c437b-0f71-4e14-bd65-6f8a0472775e None admin] RBAC: Adding query filter params (interface=public, service_id=bb3b7d15b54344b1bcd120d09589024f, region_id=RegionOne) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:56.362547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c437b-0f71-4e14-bd65-6f8a0472775e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:56.363962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c437b-0f71-4e14-bd65-6f8a0472775e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:56.368869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 655/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 12:15:56 2022] GET /identity/v3/endpoints?service_id=bb3b7d15b54344b1bcd120d09589024f&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:56.375037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:56.455303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:56.472723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783f490-190e-491e-ac4f-7aa23718bc49 None admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:56.506305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fdbbb-c0f8-4a42-8e16-b4308578b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:56.506927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fdbbb-c0f8-4a42-8e16-b4308578b1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:56.507199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fdbbb-c0f8-4a42-8e16-b4308578b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:56.507475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fdbbb-c0f8-4a42-8e16-b4308578b1f8 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:56.508072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5fdbbb-c0f8-4a42-8e16-b4308578b1f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:56.509426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5fdbbb-c0f8-4a42-8e16-b4308578b1f8 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:56.516594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 646/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 12:15:56 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:15:58.397984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954116-c5d7-4db7-8c03-50c68289880d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:58.398538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954116-c5d7-4db7-8c03-50c68289880d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:58.398538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954116-c5d7-4db7-8c03-50c68289880d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:58.399857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 656/1302] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:15:58.405805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6df5f5-7aec-4a66-a1d9-dae63704b52f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:58.406229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6df5f5-7aec-4a66-a1d9-dae63704b52f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:58.406346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6df5f5-7aec-4a66-a1d9-dae63704b52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:58.424873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ed6df5f5-7aec-4a66-a1d9-dae63704b52f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:15:58.426147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ed6df5f5-7aec-4a66-a1d9-dae63704b52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:15:58.491095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 647/1303] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:15:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:15:58.763430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954116-c5d7-4db7-8c03-50c68289880d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:58.857097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:58.863738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954116-c5d7-4db7-8c03-50c68289880d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:58.896459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:58.897045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:58.897322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:58.897592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:58.898159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:58.899530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:15:58.908417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:15:58.909580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application with Lock( Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: ERROR keystone.server.flask.application Aug 24 12:15:58.910617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 657/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 12:15:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:15:58.915562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6df5f5-7aec-4a66-a1d9-dae63704b52f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:58.988878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:15:59.010242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6df5f5-7aec-4a66-a1d9-dae63704b52f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:59.045054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084b715-8eca-4b17-ad6d-6fe97224fb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:59.045614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084b715-8eca-4b17-ad6d-6fe97224fb30 None admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:59.045888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084b715-8eca-4b17-ad6d-6fe97224fb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:59.046166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084b715-8eca-4b17-ad6d-6fe97224fb30 None admin] PATH_INFO: `/v3/services` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:59.046824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084b715-8eca-4b17-ad6d-6fe97224fb30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:15:59.047122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084b715-8eca-4b17-ad6d-6fe97224fb30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:59.048507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084b715-8eca-4b17-ad6d-6fe97224fb30 None admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:59.053523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 648/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 12:15:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:15:59.059887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:15:59.155822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504090cc-f64e-4b5a-8095-2224c6e5206e None admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:15:59.188434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e4e8a-9111-4aa2-8416-8a8c1ff88489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:15:59.189039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e4e8a-9111-4aa2-8416-8a8c1ff88489 None admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:15:59.189299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e4e8a-9111-4aa2-8416-8a8c1ff88489 None admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:15:59.189566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e4e8a-9111-4aa2-8416-8a8c1ff88489 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:15:59.190215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e4e8a-9111-4aa2-8416-8a8c1ff88489 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:15:59.191635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e4e8a-9111-4aa2-8416-8a8c1ff88489 None admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:15:59.296127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 658/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:15:59 2022] POST /identity/v3/endpoints => generated 331 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:16:09.050580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=30 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:16:09.291534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:51.909994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fc88d-f8a9-4c4d-a0cc-3edc201a2dd4 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:51.910570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fc88d-f8a9-4c4d-a0cc-3edc201a2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:51.910570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fc88d-f8a9-4c4d-a0cc-3edc201a2dd4 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:51.911913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 649/1307] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:21:51.917727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a7bb7-37c9-4cc2-8cd7-a49dbb07ce82 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:51.918174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a7bb7-37c9-4cc2-8cd7-a49dbb07ce82 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:51.918259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a7bb7-37c9-4cc2-8cd7-a49dbb07ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.005043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-104a7bb7-37c9-4cc2-8cd7-a49dbb07ce82 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:52.006668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-104a7bb7-37c9-4cc2-8cd7-a49dbb07ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:52.141921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 659/1308] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:21:52.439866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aba90-abab-4c28-a8d2-71c05500fd18 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:52.440198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aba90-abab-4c28-a8d2-71c05500fd18 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:52.440411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aba90-abab-4c28-a8d2-71c05500fd18 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.442123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 650/1309] 10.222.0.45 () {54 vars in 965 bytes} [Wed Aug 24 12:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:52.452022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b871-7a98-410b-b1fc-514236fe04ff None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:52.452519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b871-7a98-410b-b1fc-514236fe04ff None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:52.452922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b871-7a98-410b-b1fc-514236fe04ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.535165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6ca9b871-7a98-410b-b1fc-514236fe04ff None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:52.536517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6ca9b871-7a98-410b-b1fc-514236fe04ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:52.560996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:52.624074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 660/1310] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 24 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:21:52.634688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aba90-abab-4c28-a8d2-71c05500fd18 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:52.637031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c95aba90-abab-4c28-a8d2-71c05500fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:52.755032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aba90-abab-4c28-a8d2-71c05500fd18 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:52.830956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c1161-e35e-4cde-a5ea-81c18787ffa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:52.831556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c1161-e35e-4cde-a5ea-81c18787ffa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:52.831830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c1161-e35e-4cde-a5ea-81c18787ffa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:52.832102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c1161-e35e-4cde-a5ea-81c18787ffa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.833685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-732c1161-e35e-4cde-a5ea-81c18787ffa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:52.890739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:52.947177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c1161-e35e-4cde-a5ea-81c18787ffa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:52.950245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c1161-e35e-4cde-a5ea-81c18787ffa2 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:52.970251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 651/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:21:55.151983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90958703-c3fe-49a2-b30c-83b5655c4692 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:55.152572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90958703-c3fe-49a2-b30c-83b5655c4692 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:55.152572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90958703-c3fe-49a2-b30c-83b5655c4692 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:55.153962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 661/1312] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:55.159891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eba15-e1c0-44a3-85bb-d84ea6f749b2 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:55.160419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eba15-e1c0-44a3-85bb-d84ea6f749b2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:55.160419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eba15-e1c0-44a3-85bb-d84ea6f749b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:55.231112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-413eba15-e1c0-44a3-85bb-d84ea6f749b2 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:55.232311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-413eba15-e1c0-44a3-85bb-d84ea6f749b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:55.332037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 652/1313] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:21:55.610620 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90958703-c3fe-49a2-b30c-83b5655c4692 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:55.612226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-90958703-c3fe-49a2-b30c-83b5655c4692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:55.714177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90958703-c3fe-49a2-b30c-83b5655c4692 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:55.786311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:55.788815 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10227d1-2189-4437-aa0d-64e0db7964dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:55.789415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10227d1-2189-4437-aa0d-64e0db7964dd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:55.789714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10227d1-2189-4437-aa0d-64e0db7964dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:55.789981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10227d1-2189-4437-aa0d-64e0db7964dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:55.791609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e10227d1-2189-4437-aa0d-64e0db7964dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:55.898671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10227d1-2189-4437-aa0d-64e0db7964dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:55.900030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10227d1-2189-4437-aa0d-64e0db7964dd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:55.917294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 662/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:21:58.370775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116d74-24de-4be7-bd94-52cbe99c9e99 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:58.371241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116d74-24de-4be7-bd94-52cbe99c9e99 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:58.371332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116d74-24de-4be7-bd94-52cbe99c9e99 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:58.372655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 653/1315] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:58.377824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ef80-3f5e-4a0a-916a-05216761d08b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:58.378260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ef80-3f5e-4a0a-916a-05216761d08b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:58.378416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ef80-3f5e-4a0a-916a-05216761d08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:58.443184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2ff4ef80-3f5e-4a0a-916a-05216761d08b None None] MFA Rules not processed for user `796353704e344990a572797eb8638328`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:58.444401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2ff4ef80-3f5e-4a0a-916a-05216761d08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:58.497111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:58.541392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 663/1316] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:21:58.830050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae116d74-24de-4be7-bd94-52cbe99c9e99 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:58.831609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ae116d74-24de-4be7-bd94-52cbe99c9e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:58.841303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:58.932190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae116d74-24de-4be7-bd94-52cbe99c9e99 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:59.003272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40516d-29b3-417a-94f0-807a021dc7ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:59.003866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40516d-29b3-417a-94f0-807a021dc7ac service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:59.004146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40516d-29b3-417a-94f0-807a021dc7ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:59.004420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40516d-29b3-417a-94f0-807a021dc7ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:59.006012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4e40516d-29b3-417a-94f0-807a021dc7ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:59.067743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:59.113214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40516d-29b3-417a-94f0-807a021dc7ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:59.114865 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40516d-29b3-417a-94f0-807a021dc7ac service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:59.133046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 654/1317] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:21:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:01.796981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6596-ae85-4dd0-9ea6-5b1ee018db9e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:01.797515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6596-ae85-4dd0-9ea6-5b1ee018db9e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:01.797515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6596-ae85-4dd0-9ea6-5b1ee018db9e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:01.798914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 664/1318] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:01.803956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955726-413d-4e6c-8260-8a0b24497a30 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:01.804371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955726-413d-4e6c-8260-8a0b24497a30 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:01.804509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32955726-413d-4e6c-8260-8a0b24497a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:01.868894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-32955726-413d-4e6c-8260-8a0b24497a30 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:01.870609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-32955726-413d-4e6c-8260-8a0b24497a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:01.963673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 655/1319] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:02.245795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d6596-ae85-4dd0-9ea6-5b1ee018db9e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:02.247364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c65d6596-ae85-4dd0-9ea6-5b1ee018db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:02.349256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d6596-ae85-4dd0-9ea6-5b1ee018db9e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:02.421856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f537d8-52b9-4a8e-864a-ff9ac583647b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:02.422454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f537d8-52b9-4a8e-864a-ff9ac583647b service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:02.422758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f537d8-52b9-4a8e-864a-ff9ac583647b service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:02.423026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f537d8-52b9-4a8e-864a-ff9ac583647b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:02.424600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-39f537d8-52b9-4a8e-864a-ff9ac583647b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:02.432155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:02.532325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f537d8-52b9-4a8e-864a-ff9ac583647b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:02.533748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f537d8-52b9-4a8e-864a-ff9ac583647b service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:02.552162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 665/1320] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:05.493456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4da0f5-4418-4b4b-a489-7e7d0efabc0c None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:05.493971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4da0f5-4418-4b4b-a489-7e7d0efabc0c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:05.494066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4da0f5-4418-4b4b-a489-7e7d0efabc0c None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:05.495582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 656/1321] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:05.500655 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757bf3e-43d9-478d-a0f0-5e4659840f1f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:05.501064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757bf3e-43d9-478d-a0f0-5e4659840f1f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:05.501220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757bf3e-43d9-478d-a0f0-5e4659840f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:05.567865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5757bf3e-43d9-478d-a0f0-5e4659840f1f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:05.568999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5757bf3e-43d9-478d-a0f0-5e4659840f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:05.628253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:05.666450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 666/1322] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:05.960775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c1cf-d893-4da0-b3fb-7ebf5882194e None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:05.961059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c1cf-d893-4da0-b3fb-7ebf5882194e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:05.961331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c1cf-d893-4da0-b3fb-7ebf5882194e None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:05.963015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 657/1323] 10.222.0.45 () {54 vars in 965 bytes} [Wed Aug 24 12:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:05.969599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a25e4-36f8-4fb5-8936-1b8df121c2ac None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:05.969962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a25e4-36f8-4fb5-8936-1b8df121c2ac None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:05.970295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a25e4-36f8-4fb5-8936-1b8df121c2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:06.045931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3e4a25e4-36f8-4fb5-8936-1b8df121c2ac None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:06.047167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3e4a25e4-36f8-4fb5-8936-1b8df121c2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:06.125105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 667/1324] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 24 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:06.134235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c1cf-d893-4da0-b3fb-7ebf5882194e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:06.136465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6106c1cf-d893-4da0-b3fb-7ebf5882194e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:06.156944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:06.242994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c1cf-d893-4da0-b3fb-7ebf5882194e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:06.315461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d23ae-b7d6-4247-adcb-f7af0804c939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:06.316055 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d23ae-b7d6-4247-adcb-f7af0804c939 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:06.316328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d23ae-b7d6-4247-adcb-f7af0804c939 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:06.316593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d23ae-b7d6-4247-adcb-f7af0804c939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:06.318169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-831d23ae-b7d6-4247-adcb-f7af0804c939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:06.388685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:06.425033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d23ae-b7d6-4247-adcb-f7af0804c939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:06.426488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d23ae-b7d6-4247-adcb-f7af0804c939 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:06.445813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 658/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:08.591270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d821d-1893-4157-b658-f53226d94d79 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:08.591800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d821d-1893-4157-b658-f53226d94d79 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:08.591800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d821d-1893-4157-b658-f53226d94d79 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:08.593324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 668/1326] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:08.598724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9efb2-85b8-4c32-bf34-7a89ca513299 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:08.599165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9efb2-85b8-4c32-bf34-7a89ca513299 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:08.599444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9efb2-85b8-4c32-bf34-7a89ca513299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:08.670397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c1a9efb2-85b8-4c32-bf34-7a89ca513299 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:08.671650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c1a9efb2-85b8-4c32-bf34-7a89ca513299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:08.769476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 659/1327] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:09.054395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d821d-1893-4157-b658-f53226d94d79 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:09.055956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4a6d821d-1893-4157-b658-f53226d94d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:09.180887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:09.180887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d821d-1893-4157-b658-f53226d94d79 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:09.241711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9ffc0-eb25-4ce2-83ff-6d35a3d2430a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:09.242317 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9ffc0-eb25-4ce2-83ff-6d35a3d2430a service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:09.242618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9ffc0-eb25-4ce2-83ff-6d35a3d2430a service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:09.242907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9ffc0-eb25-4ce2-83ff-6d35a3d2430a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:09.244481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6da9ffc0-eb25-4ce2-83ff-6d35a3d2430a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:09.352382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:09.354730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9ffc0-eb25-4ce2-83ff-6d35a3d2430a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:09.356163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9ffc0-eb25-4ce2-83ff-6d35a3d2430a service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:09.375813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 669/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:11.326827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6291a0-2e7a-4f69-9509-10d25894910f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:11.327410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6291a0-2e7a-4f69-9509-10d25894910f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:11.327410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6291a0-2e7a-4f69-9509-10d25894910f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:11.328729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 660/1329] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:11.333857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803adff-b2ca-4ec6-9738-58c9faf96363 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:11.334292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803adff-b2ca-4ec6-9738-58c9faf96363 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:11.334402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803adff-b2ca-4ec6-9738-58c9faf96363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:11.417990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0803adff-b2ca-4ec6-9738-58c9faf96363 None None] MFA Rules not processed for user `796353704e344990a572797eb8638328`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:11.419651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0803adff-b2ca-4ec6-9738-58c9faf96363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:11.524249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 670/1330] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:11.805542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6291a0-2e7a-4f69-9509-10d25894910f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:11.807716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2a6291a0-2e7a-4f69-9509-10d25894910f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:11.837959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:11.910375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6291a0-2e7a-4f69-9509-10d25894910f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:11.986543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c533c9b-3a21-48b0-8d36-5eba13760204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:11.987223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c533c9b-3a21-48b0-8d36-5eba13760204 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:11.987500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c533c9b-3a21-48b0-8d36-5eba13760204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:11.987849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c533c9b-3a21-48b0-8d36-5eba13760204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:11.989628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1c533c9b-3a21-48b0-8d36-5eba13760204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:12.070604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:12.104647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c533c9b-3a21-48b0-8d36-5eba13760204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:12.106119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c533c9b-3a21-48b0-8d36-5eba13760204 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:12.124794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 661/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:14.429866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaf33e-eaae-4e4c-bbc7-280124d32aff None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:14.430263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaf33e-eaae-4e4c-bbc7-280124d32aff None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:14.430501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaf33e-eaae-4e4c-bbc7-280124d32aff None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:14.431982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 671/1332] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:14.438104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c83dbb-ebf4-4ac7-bdf2-f45d7b0a5dfa None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:14.438636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c83dbb-ebf4-4ac7-bdf2-f45d7b0a5dfa None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:14.438784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c83dbb-ebf4-4ac7-bdf2-f45d7b0a5dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:14.503840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-18c83dbb-ebf4-4ac7-bdf2-f45d7b0a5dfa None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:14.505031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-18c83dbb-ebf4-4ac7-bdf2-f45d7b0a5dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:14.597981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 662/1333] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:14.884533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaaf33e-eaae-4e4c-bbc7-280124d32aff None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:14.886579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cdaaf33e-eaae-4e4c-bbc7-280124d32aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:14.954090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:14.986961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaaf33e-eaae-4e4c-bbc7-280124d32aff None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:15.058483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3a907-ffd5-4eac-8ab4-00c77f9abac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:15.059102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3a907-ffd5-4eac-8ab4-00c77f9abac1 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:15.059380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3a907-ffd5-4eac-8ab4-00c77f9abac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:15.059647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3a907-ffd5-4eac-8ab4-00c77f9abac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:15.061230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3ef3a907-ffd5-4eac-8ab4-00c77f9abac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:15.168138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3a907-ffd5-4eac-8ab4-00c77f9abac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:15.169485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3a907-ffd5-4eac-8ab4-00c77f9abac1 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:15.179286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:15.187785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 672/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:17.121531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5ddf4-f88e-45ed-81de-8ec663ad0f75 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:17.129331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5ddf4-f88e-45ed-81de-8ec663ad0f75 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:17.129331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5ddf4-f88e-45ed-81de-8ec663ad0f75 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:17.129331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 663/1335] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:22:17.129623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a091d9-b336-4283-afd0-849590b84988 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:17.129623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a091d9-b336-4283-afd0-849590b84988 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:17.129623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a091d9-b336-4283-afd0-849590b84988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:17.196060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f8a091d9-b336-4283-afd0-849590b84988 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:17.197275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f8a091d9-b336-4283-afd0-849590b84988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:17.292740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 673/1336] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:17.577275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5ddf4-f88e-45ed-81de-8ec663ad0f75 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:17.578846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-78f5ddf4-f88e-45ed-81de-8ec663ad0f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:17.618740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:17.680566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5ddf4-f88e-45ed-81de-8ec663ad0f75 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:17.750991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8080c3f-d94a-4d7b-8852-2be6680343c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:17.751584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8080c3f-d94a-4d7b-8852-2be6680343c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:17.751860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8080c3f-d94a-4d7b-8852-2be6680343c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:17.752149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8080c3f-d94a-4d7b-8852-2be6680343c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:17.753696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a8080c3f-d94a-4d7b-8852-2be6680343c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:17.835293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:17.860716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8080c3f-d94a-4d7b-8852-2be6680343c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:17.862138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8080c3f-d94a-4d7b-8852-2be6680343c7 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:17.879760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 664/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:20.272113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed4fc-6796-478c-bda2-0444fe58e209 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:20.272675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed4fc-6796-478c-bda2-0444fe58e209 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:20.272789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ed4fc-6796-478c-bda2-0444fe58e209 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:20.274103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 674/1338] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:20.279351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c21f1-7a0f-4606-b8ee-f54643bf264e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:20.279751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c21f1-7a0f-4606-b8ee-f54643bf264e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:20.279929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c21f1-7a0f-4606-b8ee-f54643bf264e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:20.344022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-418c21f1-7a0f-4606-b8ee-f54643bf264e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:20.345228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-418c21f1-7a0f-4606-b8ee-f54643bf264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:20.440529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 665/1339] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:20.721561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ed4fc-6796-478c-bda2-0444fe58e209 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:20.723139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-360ed4fc-6796-478c-bda2-0444fe58e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:20.797603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:20.824446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ed4fc-6796-478c-bda2-0444fe58e209 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:20.895238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331c626-9f73-4f9e-a9a5-4105f543891e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:20.895829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331c626-9f73-4f9e-a9a5-4105f543891e service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:20.896102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331c626-9f73-4f9e-a9a5-4105f543891e service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:20.896368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331c626-9f73-4f9e-a9a5-4105f543891e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:20.897925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1331c626-9f73-4f9e-a9a5-4105f543891e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:21.005689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331c626-9f73-4f9e-a9a5-4105f543891e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:21.007147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331c626-9f73-4f9e-a9a5-4105f543891e service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:21.025083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 675/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:25.038956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc077c2-49aa-4b51-b1b0-9cba618c9d5b None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:25.039522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc077c2-49aa-4b51-b1b0-9cba618c9d5b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:25.039522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc077c2-49aa-4b51-b1b0-9cba618c9d5b None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:25.040845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 666/1341] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:25.046630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cad01-9d24-472f-ae89-815d8f3e93d4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:25.047097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cad01-9d24-472f-ae89-815d8f3e93d4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:25.047211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cad01-9d24-472f-ae89-815d8f3e93d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:25.052592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:25.121336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ee2cad01-9d24-472f-ae89-815d8f3e93d4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:25.122599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ee2cad01-9d24-472f-ae89-815d8f3e93d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:25.238240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 676/1342] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:25.519228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc077c2-49aa-4b51-b1b0-9cba618c9d5b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:25.520781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-bdc077c2-49aa-4b51-b1b0-9cba618c9d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:25.568703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:25.624124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc077c2-49aa-4b51-b1b0-9cba618c9d5b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:25.704060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca43cc-bb89-4da4-993b-5a54c494143d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:25.704584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca43cc-bb89-4da4-993b-5a54c494143d service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:25.704864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca43cc-bb89-4da4-993b-5a54c494143d service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:25.705138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca43cc-bb89-4da4-993b-5a54c494143d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:25.706787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c0ca43cc-bb89-4da4-993b-5a54c494143d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:25.805256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:25.824126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca43cc-bb89-4da4-993b-5a54c494143d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:25.825620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca43cc-bb89-4da4-993b-5a54c494143d service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:25.846192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 667/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:28.832352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbcef8-1690-4c69-950e-39fa0baec919 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:28.832857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbcef8-1690-4c69-950e-39fa0baec919 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:28.832857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbcef8-1690-4c69-950e-39fa0baec919 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:28.834273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 677/1344] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:28.839318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca492a5c-00a3-453d-865a-193f26ab201f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:28.839702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca492a5c-00a3-453d-865a-193f26ab201f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:28.839983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca492a5c-00a3-453d-865a-193f26ab201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:28.908059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ca492a5c-00a3-453d-865a-193f26ab201f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:28.909250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ca492a5c-00a3-453d-865a-193f26ab201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:29.003731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 668/1345] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:22:29.286640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbcef8-1690-4c69-950e-39fa0baec919 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:29.288267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-88bbcef8-1690-4c69-950e-39fa0baec919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:29.380711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:29.397586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbcef8-1690-4c69-950e-39fa0baec919 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:29.473914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1a972-4e6e-4e1e-a6ff-ced6124bc3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:29.474527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a972-4e6e-4e1e-a6ff-ced6124bc3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:29.474833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a972-4e6e-4e1e-a6ff-ced6124bc3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:29.475105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a972-4e6e-4e1e-a6ff-ced6124bc3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:29.476789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5fa1a972-4e6e-4e1e-a6ff-ced6124bc3c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:29.587100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1a972-4e6e-4e1e-a6ff-ced6124bc3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:29.589069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1a972-4e6e-4e1e-a6ff-ced6124bc3c7 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:29.607201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 678/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:34.240501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c9ab3-f41e-47a8-87cb-dd7d83b99dfd None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:34.241009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c9ab3-f41e-47a8-87cb-dd7d83b99dfd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:34.241102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c9ab3-f41e-47a8-87cb-dd7d83b99dfd None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:34.242410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 669/1347] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:34.248082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891d38-7dcd-442b-aac0-393623a93dda None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:34.250417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891d38-7dcd-442b-aac0-393623a93dda None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:34.250417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891d38-7dcd-442b-aac0-393623a93dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:34.264634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:34.320057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-93891d38-7dcd-442b-aac0-393623a93dda None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:34.321278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-93891d38-7dcd-442b-aac0-393623a93dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:34.425058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 679/1348] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:34.708560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c9ab3-f41e-47a8-87cb-dd7d83b99dfd None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:34.710081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6f9c9ab3-f41e-47a8-87cb-dd7d83b99dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:34.767761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:34.812425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c9ab3-f41e-47a8-87cb-dd7d83b99dfd None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:34.883720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084d623-7422-443d-acf6-d71e85d1f4a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:34.884252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084d623-7422-443d-acf6-d71e85d1f4a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:34.884551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084d623-7422-443d-acf6-d71e85d1f4a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:34.884823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084d623-7422-443d-acf6-d71e85d1f4a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:34.886451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4084d623-7422-443d-acf6-d71e85d1f4a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:34.989628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:34.993077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084d623-7422-443d-acf6-d71e85d1f4a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:34.994531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084d623-7422-443d-acf6-d71e85d1f4a8 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:35.011909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 670/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:39.862120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8104f-c510-4236-a577-92900c02265c None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:39.862759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8104f-c510-4236-a577-92900c02265c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:39.862856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8104f-c510-4236-a577-92900c02265c None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:39.865057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 680/1350] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:22:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:39.871531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52618eba-dab8-42f0-b2a5-17da4b447464 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:39.871979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52618eba-dab8-42f0-b2a5-17da4b447464 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:39.872420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52618eba-dab8-42f0-b2a5-17da4b447464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:39.964541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-52618eba-dab8-42f0-b2a5-17da4b447464 None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:39.965820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-52618eba-dab8-42f0-b2a5-17da4b447464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:40.151431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 671/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:44.246976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc94457-8d39-404f-9c58-caf46173feff None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:44.247518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc94457-8d39-404f-9c58-caf46173feff None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:44.247847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc94457-8d39-404f-9c58-caf46173feff None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:44.249361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 681/1352] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:44.254515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d74c4-0b38-41cb-a11b-34d586c4c7ee None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:44.254999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d74c4-0b38-41cb-a11b-34d586c4c7ee None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:44.255298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d74c4-0b38-41cb-a11b-34d586c4c7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:44.316062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:44.340772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-898d74c4-0b38-41cb-a11b-34d586c4c7ee None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:44.342343 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-898d74c4-0b38-41cb-a11b-34d586c4c7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:44.420285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=59 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:44.469608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 672/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:22:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:45.836609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d74656-1fa3-4433-a05a-38d84d375b19 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:45.837168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d74656-1fa3-4433-a05a-38d84d375b19 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:45.837168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d74656-1fa3-4433-a05a-38d84d375b19 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:45.838535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 682/1354] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:45.843809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a81d8-2bd8-4c50-bcd5-41ed9e65bdbd None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:45.844232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a81d8-2bd8-4c50-bcd5-41ed9e65bdbd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:45.844357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a81d8-2bd8-4c50-bcd5-41ed9e65bdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:45.921193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-701a81d8-2bd8-4c50-bcd5-41ed9e65bdbd None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:45.922391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-701a81d8-2bd8-4c50-bcd5-41ed9e65bdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:45.965659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:46.031553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 673/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:22:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:49.349255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e08515-88f5-4cf1-a8d8-59b5bdb070de None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.349720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e08515-88f5-4cf1-a8d8-59b5bdb070de None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.349812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e08515-88f5-4cf1-a8d8-59b5bdb070de None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.353285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 683/1356] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:49.362254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d4349-93f9-4a2c-b22e-af8349d3ae20 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.362758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d4349-93f9-4a2c-b22e-af8349d3ae20 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.363644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d4349-93f9-4a2c-b22e-af8349d3ae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.455802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d04d4349-93f9-4a2c-b22e-af8349d3ae20 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:49.457380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d04d4349-93f9-4a2c-b22e-af8349d3ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:49.581190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 674/1357] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:49.882984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bb0b4-b6a6-4535-aca9-1a36204550be None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.883466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bb0b4-b6a6-4535-aca9-1a36204550be None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.883936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bb0b4-b6a6-4535-aca9-1a36204550be None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.886327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 684/1358] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 12:22:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:49.893388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7047-1786-4260-8b4e-9a90316a7766 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.893605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7047-1786-4260-8b4e-9a90316a7766 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.893879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7047-1786-4260-8b4e-9a90316a7766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.914431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:49.975824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b9cf7047-1786-4260-8b4e-9a90316a7766 None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:49.977057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b9cf7047-1786-4260-8b4e-9a90316a7766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:50.102292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 675/1359] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:50.110769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bb0b4-b6a6-4535-aca9-1a36204550be None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:50.112271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-385bb0b4-b6a6-4535-aca9-1a36204550be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:50.282691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bb0b4-b6a6-4535-aca9-1a36204550be None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:50.365693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:50.405024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32ffa6-42c7-4e60-8ee5-85fff0026c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:50.405714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ffa6-42c7-4e60-8ee5-85fff0026c99 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:50.406060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ffa6-42c7-4e60-8ee5-85fff0026c99 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:50.406449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ffa6-42c7-4e60-8ee5-85fff0026c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:50.408351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4a32ffa6-42c7-4e60-8ee5-85fff0026c99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:50.502502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32ffa6-42c7-4e60-8ee5-85fff0026c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:50.504285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32ffa6-42c7-4e60-8ee5-85fff0026c99 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:50.523312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 685/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:52.652997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72ea3-7b53-42f9-91d9-368fa74b8841 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:52.653522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72ea3-7b53-42f9-91d9-368fa74b8841 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:52.653522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec72ea3-7b53-42f9-91d9-368fa74b8841 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:52.654827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 676/1361] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:52.660038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1717040-9794-4ab7-b6b6-cb10af391a8c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:52.660474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1717040-9794-4ab7-b6b6-cb10af391a8c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:52.660572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1717040-9794-4ab7-b6b6-cb10af391a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:52.732335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f1717040-9794-4ab7-b6b6-cb10af391a8c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:52.733582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f1717040-9794-4ab7-b6b6-cb10af391a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:52.773292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:52.840507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 686/1362] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:53.126140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec72ea3-7b53-42f9-91d9-368fa74b8841 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:53.127872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cec72ea3-7b53-42f9-91d9-368fa74b8841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:53.199129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:53.286266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec72ea3-7b53-42f9-91d9-368fa74b8841 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:53.406990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80355c14-6778-4cb3-9d89-73e73a32705b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:53.407543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80355c14-6778-4cb3-9d89-73e73a32705b service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:53.407859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80355c14-6778-4cb3-9d89-73e73a32705b service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:53.408090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80355c14-6778-4cb3-9d89-73e73a32705b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:53.409680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-80355c14-6778-4cb3-9d89-73e73a32705b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:53.434622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:53.507777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80355c14-6778-4cb3-9d89-73e73a32705b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:53.509741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80355c14-6778-4cb3-9d89-73e73a32705b service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:53.537923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 677/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:55.924566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b6d6-01cc-4c6d-9e80-723c2b814901 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:55.925158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b6d6-01cc-4c6d-9e80-723c2b814901 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:55.925158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b6d6-01cc-4c6d-9e80-723c2b814901 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:55.926511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 687/1364] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:55.932078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4da46-9368-4a76-9f64-64af26b225ce None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:55.932536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4da46-9368-4a76-9f64-64af26b225ce None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:55.932648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4da46-9368-4a76-9f64-64af26b225ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:56.004034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3ee4da46-9368-4a76-9f64-64af26b225ce None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:56.005243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3ee4da46-9368-4a76-9f64-64af26b225ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:56.107562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:56.114804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 678/1365] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:56.415177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a75ea-2518-49e6-85f5-52fce1699f24 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:56.415739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a75ea-2518-49e6-85f5-52fce1699f24 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:56.416169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a75ea-2518-49e6-85f5-52fce1699f24 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:56.417968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 688/1366] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 12:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:56.430136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c73fb-7e75-4c6f-985f-9265f2234cd2 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:56.430136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c73fb-7e75-4c6f-985f-9265f2234cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:56.430136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c73fb-7e75-4c6f-985f-9265f2234cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:56.514183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1c0c73fb-7e75-4c6f-985f-9265f2234cd2 None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:56.515575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1c0c73fb-7e75-4c6f-985f-9265f2234cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:56.641114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 679/1367] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:56.650173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a75ea-2518-49e6-85f5-52fce1699f24 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:56.652568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3f7a75ea-2518-49e6-85f5-52fce1699f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:56.824407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a75ea-2518-49e6-85f5-52fce1699f24 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:56.847011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:56.957876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1e3f6-67a2-429f-b097-81c6ff1bda06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:56.958700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1e3f6-67a2-429f-b097-81c6ff1bda06 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:56.959059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1e3f6-67a2-429f-b097-81c6ff1bda06 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:56.959426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1e3f6-67a2-429f-b097-81c6ff1bda06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:56.961108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-61c1e3f6-67a2-429f-b097-81c6ff1bda06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:57.052303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1e3f6-67a2-429f-b097-81c6ff1bda06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:57.054065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1e3f6-67a2-429f-b097-81c6ff1bda06 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:57.073328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 689/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:57.499876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085303f9-34c9-4044-af1d-674523424ed7 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:57.500440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085303f9-34c9-4044-af1d-674523424ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:57.500440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085303f9-34c9-4044-af1d-674523424ed7 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:57.501964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 680/1369] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:57.507481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdccf1-dc77-40e7-bea4-49670177fcea None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:57.507770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdccf1-dc77-40e7-bea4-49670177fcea None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:57.508042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdccf1-dc77-40e7-bea4-49670177fcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:57.541550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:57.577244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7dcdccf1-dc77-40e7-bea4-49670177fcea None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:57.578479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7dcdccf1-dc77-40e7-bea4-49670177fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:57.687069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 690/1370] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:58.297442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d69958-59ee-43e8-b03e-c01f7ffa4be9 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.297442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d69958-59ee-43e8-b03e-c01f7ffa4be9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.297442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d69958-59ee-43e8-b03e-c01f7ffa4be9 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.298096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 681/1371] 10.222.0.45 () {54 vars in 964 bytes} [Wed Aug 24 12:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:58.303081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132cd7b-8ec7-4cb8-a0eb-83643bd0f2e7 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.303614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132cd7b-8ec7-4cb8-a0eb-83643bd0f2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.303614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132cd7b-8ec7-4cb8-a0eb-83643bd0f2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.395989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6132cd7b-8ec7-4cb8-a0eb-83643bd0f2e7 None None] MFA Rules not processed for user `422f81200e5645668e123229a6f356d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:58.397222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6132cd7b-8ec7-4cb8-a0eb-83643bd0f2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:58.453186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.486394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 691/1372] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Aug 24 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:58.495284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d69958-59ee-43e8-b03e-c01f7ffa4be9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.496792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-69d69958-59ee-43e8-b03e-c01f7ffa4be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:58.550979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.610787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d69958-59ee-43e8-b03e-c01f7ffa4be9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:58.688315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3dd8b-bd4b-401b-8430-5f6ab61997bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:58.688959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dd8b-bd4b-401b-8430-5f6ab61997bc service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.689184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dd8b-bd4b-401b-8430-5f6ab61997bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.689458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3dd8b-bd4b-401b-8430-5f6ab61997bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.693328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3bf3dd8b-bd4b-401b-8430-5f6ab61997bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:58.794338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.807683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3dd8b-bd4b-401b-8430-5f6ab61997bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:58.809132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3dd8b-bd4b-401b-8430-5f6ab61997bc service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:58.843006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 682/1373] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:22:59.235844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d95e1-ba14-4d0c-bd9b-0cf28c49e7b1 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.236160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d95e1-ba14-4d0c-bd9b-0cf28c49e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.236438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d95e1-ba14-4d0c-bd9b-0cf28c49e7b1 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.237778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 692/1374] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:59.243448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5c36b-f2ef-4449-ac21-9166ec4d4664 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.243732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5c36b-f2ef-4449-ac21-9166ec4d4664 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.244001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5c36b-f2ef-4449-ac21-9166ec4d4664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.309987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-81d5c36b-f2ef-4449-ac21-9166ec4d4664 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:59.311344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-81d5c36b-f2ef-4449-ac21-9166ec4d4664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:59.411617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 683/1375] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:22:59.703482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d95e1-ba14-4d0c-bd9b-0cf28c49e7b1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:59.704992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-073d95e1-ba14-4d0c-bd9b-0cf28c49e7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:59.847391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d95e1-ba14-4d0c-bd9b-0cf28c49e7b1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.899037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:59.959206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935607bc-47b0-4dde-98cc-b91ffcd32702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.959803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935607bc-47b0-4dde-98cc-b91ffcd32702 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.960078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935607bc-47b0-4dde-98cc-b91ffcd32702 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.960340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935607bc-47b0-4dde-98cc-b91ffcd32702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.962038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-935607bc-47b0-4dde-98cc-b91ffcd32702 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:00.051741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935607bc-47b0-4dde-98cc-b91ffcd32702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:00.053656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935607bc-47b0-4dde-98cc-b91ffcd32702 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:00.072650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 693/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:02.183113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddea02c-0e51-4bb5-b963-840f82e0e450 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.183664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddea02c-0e51-4bb5-b963-840f82e0e450 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.183664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddea02c-0e51-4bb5-b963-840f82e0e450 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.185221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 684/1377] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:02.190266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abc354-549d-4739-b66d-8b6fca63fa7c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.190708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abc354-549d-4739-b66d-8b6fca63fa7c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.190842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abc354-549d-4739-b66d-8b6fca63fa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.267518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f5abc354-549d-4739-b66d-8b6fca63fa7c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:02.268747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f5abc354-549d-4739-b66d-8b6fca63fa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:02.275470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.382608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 694/1378] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:02.679900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddea02c-0e51-4bb5-b963-840f82e0e450 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.682058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8ddea02c-0e51-4bb5-b963-840f82e0e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:02.749082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.837542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddea02c-0e51-4bb5-b963-840f82e0e450 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.956128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33806af-7f00-4501-9783-11fc37480a7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.956834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33806af-7f00-4501-9783-11fc37480a7b service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.957219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33806af-7f00-4501-9783-11fc37480a7b service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.957593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33806af-7f00-4501-9783-11fc37480a7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.959446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c33806af-7f00-4501-9783-11fc37480a7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:02.980131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.054441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33806af-7f00-4501-9783-11fc37480a7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.056237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33806af-7f00-4501-9783-11fc37480a7b service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.075751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 685/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:05.114179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd4e42-36c7-4af2-a2fe-c1c0ca7e763b None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:05.115027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd4e42-36c7-4af2-a2fe-c1c0ca7e763b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:05.115027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd4e42-36c7-4af2-a2fe-c1c0ca7e763b None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:05.116502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 695/1380] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:05.121669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c7190-7c9e-4a4f-8778-3abe8472f0c6 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:05.122065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c7190-7c9e-4a4f-8778-3abe8472f0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:05.125066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c7190-7c9e-4a4f-8778-3abe8472f0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:05.191344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cb2c7190-7c9e-4a4f-8778-3abe8472f0c6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:05.192567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cb2c7190-7c9e-4a4f-8778-3abe8472f0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:05.310643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:05.317069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 686/1381] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:05.606511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd4e42-36c7-4af2-a2fe-c1c0ca7e763b None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:05.608048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f7bd4e42-36c7-4af2-a2fe-c1c0ca7e763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:05.761894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:05.773500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd4e42-36c7-4af2-a2fe-c1c0ca7e763b None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:05.905270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce3f57-0245-44ba-ba25-63dece75f303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:05.905879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce3f57-0245-44ba-ba25-63dece75f303 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:05.906160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce3f57-0245-44ba-ba25-63dece75f303 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:05.906432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce3f57-0245-44ba-ba25-63dece75f303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:05.908098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-54ce3f57-0245-44ba-ba25-63dece75f303 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:06.005506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce3f57-0245-44ba-ba25-63dece75f303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:06.007148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce3f57-0245-44ba-ba25-63dece75f303 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:06.020044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:06.034260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 696/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:08.166176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18fa5f-4672-4323-a106-9f28fd271386 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.166829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18fa5f-4672-4323-a106-9f28fd271386 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.166829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18fa5f-4672-4323-a106-9f28fd271386 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.168114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 687/1383] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:08.174680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcbccc-ac1e-456b-a9d6-6b9ffe02afd8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.175100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcbccc-ac1e-456b-a9d6-6b9ffe02afd8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.175238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcbccc-ac1e-456b-a9d6-6b9ffe02afd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.243111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f3bcbccc-ac1e-456b-a9d6-6b9ffe02afd8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:08.244333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f3bcbccc-ac1e-456b-a9d6-6b9ffe02afd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:08.342701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 697/1384] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:08.629817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18fa5f-4672-4323-a106-9f28fd271386 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.631359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fb18fa5f-4672-4323-a106-9f28fd271386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:08.781554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18fa5f-4672-4323-a106-9f28fd271386 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.853830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.903260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0429eb3-67b8-4739-9972-62dcb61bdbc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.903948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0429eb3-67b8-4739-9972-62dcb61bdbc0 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.904295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0429eb3-67b8-4739-9972-62dcb61bdbc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.904570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0429eb3-67b8-4739-9972-62dcb61bdbc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.906174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a0429eb3-67b8-4739-9972-62dcb61bdbc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:08.996986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0429eb3-67b8-4739-9972-62dcb61bdbc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.999148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0429eb3-67b8-4739-9972-62dcb61bdbc0 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:09.022604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 688/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:11.226083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a4e83-fb9d-45a9-b2c1-f0e193b53522 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.226776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a4e83-fb9d-45a9-b2c1-f0e193b53522 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.226776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a4e83-fb9d-45a9-b2c1-f0e193b53522 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.228033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 698/1386] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:11.233280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128399-757b-447f-b804-d0fd081b06bc None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.233883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128399-757b-447f-b804-d0fd081b06bc None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.233883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128399-757b-447f-b804-d0fd081b06bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.308215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-79128399-757b-447f-b804-d0fd081b06bc None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:11.311363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-79128399-757b-447f-b804-d0fd081b06bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:11.356118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:11.421066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 689/1387] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:23:11.713402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a4e83-fb9d-45a9-b2c1-f0e193b53522 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:11.714978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-138a4e83-fb9d-45a9-b2c1-f0e193b53522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:11.799790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:11.871426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a4e83-fb9d-45a9-b2c1-f0e193b53522 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:11.995625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f3602-e7de-4856-866e-bdb058b35083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:11.996236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f3602-e7de-4856-866e-bdb058b35083 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.996510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f3602-e7de-4856-866e-bdb058b35083 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.996780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f3602-e7de-4856-866e-bdb058b35083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.998362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b64f3602-e7de-4856-866e-bdb058b35083 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:12.036540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.088315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f3602-e7de-4856-866e-bdb058b35083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.089927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f3602-e7de-4856-866e-bdb058b35083 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.108268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 699/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:14.071907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6aec-52fa-4e01-b651-d98e29c730c5 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.072471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6aec-52fa-4e01-b651-d98e29c730c5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.072471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e6aec-52fa-4e01-b651-d98e29c730c5 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.073943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 690/1389] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:14.079009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaed8-2021-453c-84ac-4b9133d1ee67 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.079412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaed8-2021-453c-84ac-4b9133d1ee67 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.079560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaed8-2021-453c-84ac-4b9133d1ee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.146354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-dedfaed8-2021-453c-84ac-4b9133d1ee67 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:14.147581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dedfaed8-2021-453c-84ac-4b9133d1ee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:14.198443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a571-709f-4ddd-96ad-5720d8a908a3 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.198760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a571-709f-4ddd-96ad-5720d8a908a3 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.199035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a571-709f-4ddd-96ad-5720d8a908a3 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.200409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 691/1390] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:23:14.205555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439de28a-e736-4589-b4f5-02dda6d132e8 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.205844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439de28a-e736-4589-b4f5-02dda6d132e8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.206122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439de28a-e736-4589-b4f5-02dda6d132e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.244384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 700/1391] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:14.273764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-439de28a-e736-4589-b4f5-02dda6d132e8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:14.275047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-439de28a-e736-4589-b4f5-02dda6d132e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:14.369499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 692/1392] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:23:14.547372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9f8a-18b7-4cc7-add2-491c4ed757b6 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.547653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9f8a-18b7-4cc7-add2-491c4ed757b6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.547925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9f8a-18b7-4cc7-add2-491c4ed757b6 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.549312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 701/1393] 10.222.0.45 () {54 vars in 967 bytes} [Wed Aug 24 12:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:14.554340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1350a-aafa-478f-8a87-cf63a6ae91fb None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.554625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1350a-aafa-478f-8a87-cf63a6ae91fb None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.554935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1350a-aafa-478f-8a87-cf63a6ae91fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.571453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:14.630921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e1a1350a-aafa-478f-8a87-cf63a6ae91fb None None] MFA Rules not processed for user `2f33e1c573c044f086f99eb128c8f726`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:14.632058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e1a1350a-aafa-478f-8a87-cf63a6ae91fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:14.659968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda9f8a-18b7-4cc7-add2-491c4ed757b6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:14.661714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0cda9f8a-18b7-4cc7-add2-491c4ed757b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:14.675601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:14.709210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 693/1394] 10.222.0.45 () {58 vars in 1064 bytes} [Wed Aug 24 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:14.716782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1350a-aafa-478f-8a87-cf63a6ae91fb None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:14.724174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1350a-aafa-478f-8a87-cf63a6ae91fb None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:14.798350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29426e75-748c-40ad-b59e-6d2535bd94c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f33e1c573c044f086f99eb128c8f726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:14.798979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29426e75-748c-40ad-b59e-6d2535bd94c7 service glance] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.799257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29426e75-748c-40ad-b59e-6d2535bd94c7 service glance] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.799527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29426e75-748c-40ad-b59e-6d2535bd94c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.801119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-29426e75-748c-40ad-b59e-6d2535bd94c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:14.812102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda9f8a-18b7-4cc7-add2-491c4ed757b6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:14.831194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:14.901304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:14.914137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29426e75-748c-40ad-b59e-6d2535bd94c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:14.916177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29426e75-748c-40ad-b59e-6d2535bd94c7 service glance] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:14.928242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000883d6-9882-42e2-8827-29382a12474b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:14.928839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000883d6-9882-42e2-8827-29382a12474b service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:14.929113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000883d6-9882-42e2-8827-29382a12474b service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:14.929391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000883d6-9882-42e2-8827-29382a12474b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:14.931055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-000883d6-9882-42e2-8827-29382a12474b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:14.934549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 694/1395] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:23:15.020650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000883d6-9882-42e2-8827-29382a12474b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:15.022255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000883d6-9882-42e2-8827-29382a12474b service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:15.041425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 702/1396] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:23:15.815031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b2d08-76a0-424d-bdf4-012fa6295281 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:15.815515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b2d08-76a0-424d-bdf4-012fa6295281 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:15.815719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b2d08-76a0-424d-bdf4-012fa6295281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:15.883949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0c5b2d08-76a0-424d-bdf4-012fa6295281 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:15.885182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0c5b2d08-76a0-424d-bdf4-012fa6295281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:15.968338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:15.974251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 695/1397] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:16.018087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e38fe1-81c9-4b2d-b4b7-89ad81b5ae05 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.018577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e38fe1-81c9-4b2d-b4b7-89ad81b5ae05 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.018720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e38fe1-81c9-4b2d-b4b7-89ad81b5ae05 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.020019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 703/1398] 10.222.0.45 () {54 vars in 943 bytes} [Wed Aug 24 12:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:16.026872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f45ee-02e8-4bfb-9ede-6f92b92937db None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.027163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f45ee-02e8-4bfb-9ede-6f92b92937db None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.027676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f45ee-02e8-4bfb-9ede-6f92b92937db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.102396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1a2f45ee-02e8-4bfb-9ede-6f92b92937db None None] MFA Rules not processed for user `909708a2a44a48e7b62a6f6a8d792f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:16.103567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1a2f45ee-02e8-4bfb-9ede-6f92b92937db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:16.213960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 696/1399] 10.222.0.45 () {58 vars in 1040 bytes} [Wed Aug 24 12:23:16 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:16.223495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e38fe1-81c9-4b2d-b4b7-89ad81b5ae05 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.225005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-37e38fe1-81c9-4b2d-b4b7-89ad81b5ae05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:16.320517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.361384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e38fe1-81c9-4b2d-b4b7-89ad81b5ae05 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.474605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a641225-d9f8-4f60-a23e-d201a00331fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.475844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a641225-d9f8-4f60-a23e-d201a00331fa service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.476132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a641225-d9f8-4f60-a23e-d201a00331fa service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.476405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a641225-d9f8-4f60-a23e-d201a00331fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.478064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0a641225-d9f8-4f60-a23e-d201a00331fa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:16.560438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a641225-d9f8-4f60-a23e-d201a00331fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.562028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a641225-d9f8-4f60-a23e-d201a00331fa service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.565665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.571406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 704/1400] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:23:17.234984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175744-ac92-4510-a32c-48fdc788343d None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.235507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175744-ac92-4510-a32c-48fdc788343d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.235856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175744-ac92-4510-a32c-48fdc788343d None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.237560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 697/1401] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:17.243820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97363487-ec9e-4365-a691-81f43c8e22a2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.244245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97363487-ec9e-4365-a691-81f43c8e22a2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.244505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97363487-ec9e-4365-a691-81f43c8e22a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.321604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-97363487-ec9e-4365-a691-81f43c8e22a2 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:17.322994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-97363487-ec9e-4365-a691-81f43c8e22a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:17.431770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 705/1402] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:17.723834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175744-ac92-4510-a32c-48fdc788343d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.725742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7d175744-ac92-4510-a32c-48fdc788343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:17.790233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.892774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175744-ac92-4510-a32c-48fdc788343d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:18.041184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733caff9-eadd-4efb-90c1-67b76a583111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:18.041997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733caff9-eadd-4efb-90c1-67b76a583111 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:18.042374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733caff9-eadd-4efb-90c1-67b76a583111 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:18.042791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733caff9-eadd-4efb-90c1-67b76a583111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:18.044561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-733caff9-eadd-4efb-90c1-67b76a583111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:18.071332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:18.168145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733caff9-eadd-4efb-90c1-67b76a583111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:18.169798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733caff9-eadd-4efb-90c1-67b76a583111 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:18.190702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 698/1403] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:20.326242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663e58c-81ba-4084-b21c-f24e3bff3fda None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.326886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663e58c-81ba-4084-b21c-f24e3bff3fda None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.326886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663e58c-81ba-4084-b21c-f24e3bff3fda None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.328186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 706/1404] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:20.333955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26649b7a-83b6-4bae-8ecf-71ab8a0ac58a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.334875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26649b7a-83b6-4bae-8ecf-71ab8a0ac58a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.335406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26649b7a-83b6-4bae-8ecf-71ab8a0ac58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.417468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-26649b7a-83b6-4bae-8ecf-71ab8a0ac58a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:20.418743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-26649b7a-83b6-4bae-8ecf-71ab8a0ac58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:20.529960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:20.537724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 699/1405] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:20.850181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663e58c-81ba-4084-b21c-f24e3bff3fda None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:20.851799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9663e58c-81ba-4084-b21c-f24e3bff3fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:20.940728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:21.015332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663e58c-81ba-4084-b21c-f24e3bff3fda None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:21.148430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ccdb9-91b1-4bda-9bb7-dbcd4ab872b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:21.148973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ccdb9-91b1-4bda-9bb7-dbcd4ab872b9 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:21.149250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ccdb9-91b1-4bda-9bb7-dbcd4ab872b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:21.149521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ccdb9-91b1-4bda-9bb7-dbcd4ab872b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:21.151169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-711ccdb9-91b1-4bda-9bb7-dbcd4ab872b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:21.246303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:21.267841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ccdb9-91b1-4bda-9bb7-dbcd4ab872b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:21.269444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711ccdb9-91b1-4bda-9bb7-dbcd4ab872b9 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:21.292396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 707/1406] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:23.346985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a749290-6205-4adb-9e57-91acb450d70c None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:23.347554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a749290-6205-4adb-9e57-91acb450d70c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:23.347554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a749290-6205-4adb-9e57-91acb450d70c None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:23.348863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 700/1407] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:23.354294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ce8a8-f8fb-401b-82de-6e25f745180c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:23.354754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ce8a8-f8fb-401b-82de-6e25f745180c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:23.354871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ce8a8-f8fb-401b-82de-6e25f745180c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:23.428768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fa8ce8a8-f8fb-401b-82de-6e25f745180c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:23.430720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fa8ce8a8-f8fb-401b-82de-6e25f745180c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:23.540196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 708/1408] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:23.834607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a749290-6205-4adb-9e57-91acb450d70c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:23.836478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9a749290-6205-4adb-9e57-91acb450d70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:24.024278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a749290-6205-4adb-9e57-91acb450d70c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:24.097142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:24.150109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad29d8b-e033-42c1-9484-4eb7c4034228 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:24.150765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad29d8b-e033-42c1-9484-4eb7c4034228 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:24.151032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad29d8b-e033-42c1-9484-4eb7c4034228 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:24.151458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad29d8b-e033-42c1-9484-4eb7c4034228 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:24.153049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fad29d8b-e033-42c1-9484-4eb7c4034228 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:24.251703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad29d8b-e033-42c1-9484-4eb7c4034228 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:24.253341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad29d8b-e033-42c1-9484-4eb7c4034228 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:24.272033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 701/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:33.538283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:34.269999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=65 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:40.236443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41917bc2-e954-480c-b6e1-f6cbba4841f1 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:40.237021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41917bc2-e954-480c-b6e1-f6cbba4841f1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:40.237021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41917bc2-e954-480c-b6e1-f6cbba4841f1 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:40.238358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 709/1410] 10.222.0.45 () {54 vars in 918 bytes} [Wed Aug 24 12:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:40.244021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36125-a499-46cd-8f4a-8c412f8eba46 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:40.244406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36125-a499-46cd-8f4a-8c412f8eba46 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:40.244570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36125-a499-46cd-8f4a-8c412f8eba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:40.318922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-69c36125-a499-46cd-8f4a-8c412f8eba46 None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:40.320678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-69c36125-a499-46cd-8f4a-8c412f8eba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:40.424584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 702/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 24 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:40.439428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ca76-a80b-47df-b1fe-964d3ea9eae5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:40.439711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ca76-a80b-47df-b1fe-964d3ea9eae5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:40.439984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ca76-a80b-47df-b1fe-964d3ea9eae5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:40.441294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 710/1412] 10.222.0.45 () {54 vars in 918 bytes} [Wed Aug 24 12:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:40.446431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9813b2-6462-4ed5-814b-9ba13e66ce5d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:40.446742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9813b2-6462-4ed5-814b-9ba13e66ce5d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:40.447018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9813b2-6462-4ed5-814b-9ba13e66ce5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:40.517249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:40.520236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3d9813b2-6462-4ed5-814b-9ba13e66ce5d None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:40.521426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3d9813b2-6462-4ed5-814b-9ba13e66ce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:40.636675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 703/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 24 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:43.003553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354492-6920-4803-bbc3-a918f9a25ba5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:43.004118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354492-6920-4803-bbc3-a918f9a25ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:43.004118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4354492-6920-4803-bbc3-a918f9a25ba5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:43.005469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:43.011280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5bf8-265c-47be-8e75-a4c93182cb22 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:43.011733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5bf8-265c-47be-8e75-a4c93182cb22 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:43.011848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5bf8-265c-47be-8e75-a4c93182cb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:43.075769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-15be5bf8-265c-47be-8e75-a4c93182cb22 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:43.076969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-15be5bf8-265c-47be-8e75-a4c93182cb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:43.130121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:43.171575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:43.458284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354492-6920-4803-bbc3-a918f9a25ba5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:43.459914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c4354492-6920-4803-bbc3-a918f9a25ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:43.601306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4354492-6920-4803-bbc3-a918f9a25ba5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:43.671527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:43.710720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1736c6-2fee-4534-8fae-31ff551d80e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:43.711255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1736c6-2fee-4534-8fae-31ff551d80e3 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:43.711532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1736c6-2fee-4534-8fae-31ff551d80e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:43.711797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1736c6-2fee-4534-8fae-31ff551d80e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:43.713404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ac1736c6-2fee-4534-8fae-31ff551d80e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:43.801754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1736c6-2fee-4534-8fae-31ff551d80e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:43.803392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1736c6-2fee-4534-8fae-31ff551d80e3 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:43.821691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 712/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:44.707254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4be34-34ab-45dd-ac71-a8348b6e6a9a None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:44.707764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4be34-34ab-45dd-ac71-a8348b6e6a9a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:44.707764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4be34-34ab-45dd-ac71-a8348b6e6a9a None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:44.709064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:44.714019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e179e8-ee9f-4494-a9a9-573f1fff7222 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:44.714396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e179e8-ee9f-4494-a9a9-573f1fff7222 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:44.714571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e179e8-ee9f-4494-a9a9-573f1fff7222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:44.789812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e6e179e8-ee9f-4494-a9a9-573f1fff7222 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:44.791041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e6e179e8-ee9f-4494-a9a9-573f1fff7222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:44.817588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:44.887444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:45.035762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4be34-34ab-45dd-ac71-a8348b6e6a9a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:45.037328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b6a4be34-34ab-45dd-ac71-a8348b6e6a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:45.180140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4be34-34ab-45dd-ac71-a8348b6e6a9a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:45.219298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:45.292211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7aa86-6f15-4e55-adce-7c3d2a411e1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:45.292725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7aa86-6f15-4e55-adce-7c3d2a411e1a service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:45.293001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7aa86-6f15-4e55-adce-7c3d2a411e1a service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:45.293267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7aa86-6f15-4e55-adce-7c3d2a411e1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:45.294883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-77e7aa86-6f15-4e55-adce-7c3d2a411e1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:45.383118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7aa86-6f15-4e55-adce-7c3d2a411e1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:45.384720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7aa86-6f15-4e55-adce-7c3d2a411e1a service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:45.402155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 706/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:46.282103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079d364-5d8a-4ee7-916e-f92a318b5a90 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:46.282697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079d364-5d8a-4ee7-916e-f92a318b5a90 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:46.282697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079d364-5d8a-4ee7-916e-f92a318b5a90 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:46.284004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:46.289612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b17e51-7056-4649-b8b4-c4682e118641 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:46.290027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b17e51-7056-4649-b8b4-c4682e118641 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:46.290135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b17e51-7056-4649-b8b4-c4682e118641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:46.344662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:46.366121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e0b17e51-7056-4649-b8b4-c4682e118641 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:46.367407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e0b17e51-7056-4649-b8b4-c4682e118641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:46.463148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:46.616947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079d364-5d8a-4ee7-916e-f92a318b5a90 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:46.618369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c079d364-5d8a-4ee7-916e-f92a318b5a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:46.760910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079d364-5d8a-4ee7-916e-f92a318b5a90 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:46.772909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:46.872243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f596d2-f498-4704-8890-eb3c27e7f808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:46.872756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f596d2-f498-4704-8890-eb3c27e7f808 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:46.873029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f596d2-f498-4704-8890-eb3c27e7f808 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:46.873294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f596d2-f498-4704-8890-eb3c27e7f808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:46.874885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-52f596d2-f498-4704-8890-eb3c27e7f808 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:46.961075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f596d2-f498-4704-8890-eb3c27e7f808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:46.962673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f596d2-f498-4704-8890-eb3c27e7f808 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:46.980874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 715/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:47.832992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa9274-3fdd-4ebf-af16-41831783fb5f None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:47.833520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa9274-3fdd-4ebf-af16-41831783fb5f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:47.833520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa9274-3fdd-4ebf-af16-41831783fb5f None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:47.834905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:47.839885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a310-d292-443e-bf37-fb5af1bf8000 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:47.840310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a310-d292-443e-bf37-fb5af1bf8000 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:47.840415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a310-d292-443e-bf37-fb5af1bf8000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:47.860023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:47.916120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7b07a310-d292-443e-bf37-fb5af1bf8000 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:47.917328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7b07a310-d292-443e-bf37-fb5af1bf8000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:48.013970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:48.163886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa9274-3fdd-4ebf-af16-41831783fb5f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:48.165337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cbaa9274-3fdd-4ebf-af16-41831783fb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:48.272154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:48.307035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa9274-3fdd-4ebf-af16-41831783fb5f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:48.417702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5a9c8-3a0e-4ffa-a989-b71fdddb4239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:48.418222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5a9c8-3a0e-4ffa-a989-b71fdddb4239 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:48.418495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5a9c8-3a0e-4ffa-a989-b71fdddb4239 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:48.418794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5a9c8-3a0e-4ffa-a989-b71fdddb4239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:48.420362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8cb5a9c8-3a0e-4ffa-a989-b71fdddb4239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:48.494628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:48.508113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5a9c8-3a0e-4ffa-a989-b71fdddb4239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:48.509690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5a9c8-3a0e-4ffa-a989-b71fdddb4239 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:48.527930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 709/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:49.383493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef34883-679f-4b59-8dbf-5cd79c41b54e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:49.384065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef34883-679f-4b59-8dbf-5cd79c41b54e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:49.384065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef34883-679f-4b59-8dbf-5cd79c41b54e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:49.385460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:49.390348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89065f41-ff86-4716-84ed-cb2fa5966b01 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:49.390790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89065f41-ff86-4716-84ed-cb2fa5966b01 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:49.390927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89065f41-ff86-4716-84ed-cb2fa5966b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:49.466333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-89065f41-ff86-4716-84ed-cb2fa5966b01 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:49.467570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-89065f41-ff86-4716-84ed-cb2fa5966b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:49.561069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:49.712185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef34883-679f-4b59-8dbf-5cd79c41b54e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:49.713603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-eef34883-679f-4b59-8dbf-5cd79c41b54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:49.795999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:49.856334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef34883-679f-4b59-8dbf-5cd79c41b54e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:49.967645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b04b99-a532-4931-8591-0ce01d8858f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:49.968151 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b04b99-a532-4931-8591-0ce01d8858f8 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:49.968424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b04b99-a532-4931-8591-0ce01d8858f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:49.968692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b04b99-a532-4931-8591-0ce01d8858f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:49.970269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f9b04b99-a532-4931-8591-0ce01d8858f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:50.018807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:50.060672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b04b99-a532-4931-8591-0ce01d8858f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:50.062624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b04b99-a532-4931-8591-0ce01d8858f8 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:50.081013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 718/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:50.951053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f96f-fbf0-4e41-8a5c-f069c944a5eb None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:50.951615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f96f-fbf0-4e41-8a5c-f069c944a5eb None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:50.951615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434f96f-fbf0-4e41-8a5c-f069c944a5eb None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:50.952957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:50.957993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04467f-897d-4bd5-b100-8e6aaf34dcb4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:50.958396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04467f-897d-4bd5-b100-8e6aaf34dcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:50.958577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04467f-897d-4bd5-b100-8e6aaf34dcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:51.033030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2b04467f-897d-4bd5-b100-8e6aaf34dcb4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:51.034213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2b04467f-897d-4bd5-b100-8e6aaf34dcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:51.131567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:51.286270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434f96f-fbf0-4e41-8a5c-f069c944a5eb None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:51.287823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3434f96f-fbf0-4e41-8a5c-f069c944a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:51.336778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:51.434787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3434f96f-fbf0-4e41-8a5c-f069c944a5eb None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:51.545953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad757cfa-99f4-46a6-9480-22c904a2aea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:51.546545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad757cfa-99f4-46a6-9480-22c904a2aea8 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:51.546858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad757cfa-99f4-46a6-9480-22c904a2aea8 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:51.547129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad757cfa-99f4-46a6-9480-22c904a2aea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:51.548724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ad757cfa-99f4-46a6-9480-22c904a2aea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:51.555952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:51.636808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad757cfa-99f4-46a6-9480-22c904a2aea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:51.638380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad757cfa-99f4-46a6-9480-22c904a2aea8 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:51.655850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 712/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:23:52.573107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad894b-1d8f-45c3-aa8b-f22ec7407982 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:52.573666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad894b-1d8f-45c3-aa8b-f22ec7407982 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:52.573666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad894b-1d8f-45c3-aa8b-f22ec7407982 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:52.575022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:52.579944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5122210-c774-43fa-a675-f907677f28c4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:52.580384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5122210-c774-43fa-a675-f907677f28c4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:52.580496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5122210-c774-43fa-a675-f907677f28c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:52.653949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d5122210-c774-43fa-a675-f907677f28c4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:52.655169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d5122210-c774-43fa-a675-f907677f28c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:52.721862 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:52.750978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:52.905864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad894b-1d8f-45c3-aa8b-f22ec7407982 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:52.907326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-35ad894b-1d8f-45c3-aa8b-f22ec7407982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:52.924934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:53.052121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad894b-1d8f-45c3-aa8b-f22ec7407982 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:53.137391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:53.162194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323332fd-f08a-48bf-9517-93015b85d636 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:53.162766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323332fd-f08a-48bf-9517-93015b85d636 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:53.163122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323332fd-f08a-48bf-9517-93015b85d636 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:53.163392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323332fd-f08a-48bf-9517-93015b85d636 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:53.164969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-323332fd-f08a-48bf-9517-93015b85d636 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:53.251827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323332fd-f08a-48bf-9517-93015b85d636 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:53.253419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323332fd-f08a-48bf-9517-93015b85d636 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:53.271398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 721/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:54.128800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c16a-370b-48f1-9872-d5a2b9e41390 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:54.129385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c16a-370b-48f1-9872-d5a2b9e41390 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:54.129385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c16a-370b-48f1-9872-d5a2b9e41390 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:54.130738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:54.138102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89660-b583-47e6-b1ea-5bda139a1209 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:54.138680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89660-b583-47e6-b1ea-5bda139a1209 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:54.138680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89660-b583-47e6-b1ea-5bda139a1209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:54.213462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bbb89660-b583-47e6-b1ea-5bda139a1209 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:54.214570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bbb89660-b583-47e6-b1ea-5bda139a1209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:54.257170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:54.428206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:54.579713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751c16a-370b-48f1-9872-d5a2b9e41390 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:54.581215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0751c16a-370b-48f1-9872-d5a2b9e41390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:54.721827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751c16a-370b-48f1-9872-d5a2b9e41390 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:54.774806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:54.831823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ccac1-1cff-4dc8-8fe4-b7f40dd24cf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:54.832351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ccac1-1cff-4dc8-8fe4-b7f40dd24cf6 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:54.832627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ccac1-1cff-4dc8-8fe4-b7f40dd24cf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:54.832895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ccac1-1cff-4dc8-8fe4-b7f40dd24cf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:54.834478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a41ccac1-1cff-4dc8-8fe4-b7f40dd24cf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:54.921258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ccac1-1cff-4dc8-8fe4-b7f40dd24cf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:54.922868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ccac1-1cff-4dc8-8fe4-b7f40dd24cf6 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:54.940701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 715/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:55.808798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eed6b7-381c-4c72-8ecc-dc4eea9dff7a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:55.808798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eed6b7-381c-4c72-8ecc-dc4eea9dff7a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:55.808798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eed6b7-381c-4c72-8ecc-dc4eea9dff7a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:55.808798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:55.811241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1019c-be3e-4aac-99e9-377895b4f67d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:55.811720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1019c-be3e-4aac-99e9-377895b4f67d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:55.811885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1019c-be3e-4aac-99e9-377895b4f67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:55.886320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0af1019c-be3e-4aac-99e9-377895b4f67d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:55.887570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0af1019c-be3e-4aac-99e9-377895b4f67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:55.889582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:55.982575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:56.133786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eed6b7-381c-4c72-8ecc-dc4eea9dff7a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:56.135299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c4eed6b7-381c-4c72-8ecc-dc4eea9dff7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:56.280691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eed6b7-381c-4c72-8ecc-dc4eea9dff7a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:56.307120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:56.391681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65652f0b-5aeb-4cb4-beee-84ecead903b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:56.392246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65652f0b-5aeb-4cb4-beee-84ecead903b6 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:56.392521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65652f0b-5aeb-4cb4-beee-84ecead903b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:56.392790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65652f0b-5aeb-4cb4-beee-84ecead903b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:56.394360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-65652f0b-5aeb-4cb4-beee-84ecead903b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:56.484171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65652f0b-5aeb-4cb4-beee-84ecead903b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:56.485742 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65652f0b-5aeb-4cb4-beee-84ecead903b6 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:56.503691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 724/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:23:58.300427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d08122-3ac9-4629-9482-fd733f2413fc None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:58.301011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d08122-3ac9-4629-9482-fd733f2413fc None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:58.301011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d08122-3ac9-4629-9482-fd733f2413fc None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:58.302349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:58.308069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9947d0-2186-4702-b161-10780371cf8c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:58.308520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9947d0-2186-4702-b161-10780371cf8c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:58.308636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9947d0-2186-4702-b161-10780371cf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:58.348950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:58.373993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8d9947d0-2186-4702-b161-10780371cf8c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:58.375234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8d9947d0-2186-4702-b161-10780371cf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:58.472891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:23:58.758195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d08122-3ac9-4629-9482-fd733f2413fc None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:58.759753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f3d08122-3ac9-4629-9482-fd733f2413fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:58.860120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d08122-3ac9-4629-9482-fd733f2413fc None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:58.891317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:58.931194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78aeb78-1e25-4923-b64d-385fdd8e9cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:58.931789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aeb78-1e25-4923-b64d-385fdd8e9cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:58.932063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aeb78-1e25-4923-b64d-385fdd8e9cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:58.932329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aeb78-1e25-4923-b64d-385fdd8e9cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:58.933898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f78aeb78-1e25-4923-b64d-385fdd8e9cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:59.040716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78aeb78-1e25-4923-b64d-385fdd8e9cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:59.042081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78aeb78-1e25-4923-b64d-385fdd8e9cf3 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:59.062372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 718/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:01.101336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d274-0d0d-4a77-9855-84082b0e6a27 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:01.101908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d274-0d0d-4a77-9855-84082b0e6a27 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:01.101908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2d274-0d0d-4a77-9855-84082b0e6a27 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:01.103231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:01.108508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc0aac-a1fd-4326-aea3-fecf98e57b0c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:01.108889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc0aac-a1fd-4326-aea3-fecf98e57b0c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:01.109045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc0aac-a1fd-4326-aea3-fecf98e57b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:01.173863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-07cc0aac-a1fd-4326-aea3-fecf98e57b0c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:01.175111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-07cc0aac-a1fd-4326-aea3-fecf98e57b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:01.187575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:01.271728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:01.555333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d274-0d0d-4a77-9855-84082b0e6a27 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:01.556875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-78b2d274-0d0d-4a77-9855-84082b0e6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:01.656641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2d274-0d0d-4a77-9855-84082b0e6a27 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:01.668344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:01.728439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e875fc-a5a2-4007-820d-9fcacb5508b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:01.728957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e875fc-a5a2-4007-820d-9fcacb5508b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:01.729233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e875fc-a5a2-4007-820d-9fcacb5508b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:01.729504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e875fc-a5a2-4007-820d-9fcacb5508b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:01.731119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-54e875fc-a5a2-4007-820d-9fcacb5508b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:01.836372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e875fc-a5a2-4007-820d-9fcacb5508b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:01.837798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e875fc-a5a2-4007-820d-9fcacb5508b0 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:01.856309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 727/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:04.150217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44883b5-6447-4b38-91ec-d91cfda5b1b9 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:04.150800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44883b5-6447-4b38-91ec-d91cfda5b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:04.150899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44883b5-6447-4b38-91ec-d91cfda5b1b9 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:04.152202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:04.157583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c86f1-32de-4b22-81fc-ba235563ab37 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:04.157977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c86f1-32de-4b22-81fc-ba235563ab37 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:04.158129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c86f1-32de-4b22-81fc-ba235563ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:04.205246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:04.223050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1a1c86f1-32de-4b22-81fc-ba235563ab37 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:04.225879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1a1c86f1-32de-4b22-81fc-ba235563ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:04.319539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:04.607311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44883b5-6447-4b38-91ec-d91cfda5b1b9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:04.608852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f44883b5-6447-4b38-91ec-d91cfda5b1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:04.712110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44883b5-6447-4b38-91ec-d91cfda5b1b9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:04.751129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:04.784798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83a1cc-3890-473c-9cbb-863029ac0b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:04.785403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83a1cc-3890-473c-9cbb-863029ac0b6e service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:04.785683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83a1cc-3890-473c-9cbb-863029ac0b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:04.785955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83a1cc-3890-473c-9cbb-863029ac0b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:04.787594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9c83a1cc-3890-473c-9cbb-863029ac0b6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:04.894327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83a1cc-3890-473c-9cbb-863029ac0b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:04.895789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83a1cc-3890-473c-9cbb-863029ac0b6e service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:04.914079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 721/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:06.929361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f488d4a-14ed-4e93-943f-2aaa20b1f50e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:06.929974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f488d4a-14ed-4e93-943f-2aaa20b1f50e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:06.929974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f488d4a-14ed-4e93-943f-2aaa20b1f50e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:06.931270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:06.936631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bb54b-0315-46d9-9f98-723363bb8dce None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:06.937069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bb54b-0315-46d9-9f98-723363bb8dce None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:06.937178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bb54b-0315-46d9-9f98-723363bb8dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:07.004036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e95bb54b-0315-46d9-9f98-723363bb8dce None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:07.005238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e95bb54b-0315-46d9-9f98-723363bb8dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:07.026154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:07.100346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:07.383511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f488d4a-14ed-4e93-943f-2aaa20b1f50e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:07.385141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4f488d4a-14ed-4e93-943f-2aaa20b1f50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:07.490106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f488d4a-14ed-4e93-943f-2aaa20b1f50e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:07.510352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:07.562859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392a17a-addf-4a98-8c7c-792e028865b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:07.563454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a17a-addf-4a98-8c7c-792e028865b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:07.563732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a17a-addf-4a98-8c7c-792e028865b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:07.564000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a17a-addf-4a98-8c7c-792e028865b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:07.565575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2392a17a-addf-4a98-8c7c-792e028865b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:07.672027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392a17a-addf-4a98-8c7c-792e028865b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:07.673378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392a17a-addf-4a98-8c7c-792e028865b3 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:07.691447 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 730/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:09.640169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d2b62-32be-4cea-a7db-1f4537460d52 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:09.640731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d2b62-32be-4cea-a7db-1f4537460d52 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:09.640731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d2b62-32be-4cea-a7db-1f4537460d52 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:09.642042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:09.647454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f52c68-8a0d-48c1-b90f-07cdec3718a9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:09.647903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f52c68-8a0d-48c1-b90f-07cdec3718a9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:09.648001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f52c68-8a0d-48c1-b90f-07cdec3718a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:09.709627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:09.712211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-63f52c68-8a0d-48c1-b90f-07cdec3718a9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:09.713397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-63f52c68-8a0d-48c1-b90f-07cdec3718a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:09.807688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:10.093749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d2b62-32be-4cea-a7db-1f4537460d52 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:10.093749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-738d2b62-32be-4cea-a7db-1f4537460d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:10.196011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d2b62-32be-4cea-a7db-1f4537460d52 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:10.245180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:10.268815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230e28b-ac3f-400d-a526-e1f45c2de07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:10.269419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230e28b-ac3f-400d-a526-e1f45c2de07a service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:10.269693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230e28b-ac3f-400d-a526-e1f45c2de07a service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:10.269960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230e28b-ac3f-400d-a526-e1f45c2de07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:10.271619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0230e28b-ac3f-400d-a526-e1f45c2de07a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:10.379195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230e28b-ac3f-400d-a526-e1f45c2de07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:10.380565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230e28b-ac3f-400d-a526-e1f45c2de07a service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:10.399236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 724/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:12.348337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc915-310a-4640-a9bc-c5a40f57e17d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:12.348911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc915-310a-4640-a9bc-c5a40f57e17d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:12.348911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc915-310a-4640-a9bc-c5a40f57e17d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:12.350785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:12.356269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727328d-5998-47df-b031-6e6869f668ca None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:12.356668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727328d-5998-47df-b031-6e6869f668ca None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:12.356815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727328d-5998-47df-b031-6e6869f668ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:12.423798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7727328d-5998-47df-b031-6e6869f668ca None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:12.424997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7727328d-5998-47df-b031-6e6869f668ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:12.453934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:12.520026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:12.813550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bc915-310a-4640-a9bc-c5a40f57e17d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:12.815113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1a6bc915-310a-4640-a9bc-c5a40f57e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:12.921158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bc915-310a-4640-a9bc-c5a40f57e17d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:12.950549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:12.995241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98750ae-69bb-48e5-997e-74c53cd0cb69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:12.995827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98750ae-69bb-48e5-997e-74c53cd0cb69 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:12.996099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98750ae-69bb-48e5-997e-74c53cd0cb69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:12.996363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98750ae-69bb-48e5-997e-74c53cd0cb69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:12.997926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b98750ae-69bb-48e5-997e-74c53cd0cb69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:13.101836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98750ae-69bb-48e5-997e-74c53cd0cb69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:13.103283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98750ae-69bb-48e5-997e-74c53cd0cb69 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:13.120796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 733/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:15.072500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d33c4-8981-4cb5-88e9-5c206d722ea6 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:15.073075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d33c4-8981-4cb5-88e9-5c206d722ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:15.073075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d33c4-8981-4cb5-88e9-5c206d722ea6 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:15.074434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:15.080000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1549f0-933a-4f00-9a5a-3eee1503fd48 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:15.080423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1549f0-933a-4f00-9a5a-3eee1503fd48 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:15.080528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1549f0-933a-4f00-9a5a-3eee1503fd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:15.145486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9c1549f0-933a-4f00-9a5a-3eee1503fd48 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:15.146722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9c1549f0-933a-4f00-9a5a-3eee1503fd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:15.154838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:15.242467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:15.532038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d33c4-8981-4cb5-88e9-5c206d722ea6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:15.533563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cf0d33c4-8981-4cb5-88e9-5c206d722ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:15.636299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d33c4-8981-4cb5-88e9-5c206d722ea6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:15.688667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:15.706853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4c553-b4ce-4e4d-92f1-f49f22ef000c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:15.707444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c553-b4ce-4e4d-92f1-f49f22ef000c service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:15.707716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c553-b4ce-4e4d-92f1-f49f22ef000c service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:15.707977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c553-b4ce-4e4d-92f1-f49f22ef000c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:15.709563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-86d4c553-b4ce-4e4d-92f1-f49f22ef000c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:15.814384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4c553-b4ce-4e4d-92f1-f49f22ef000c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:15.815855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4c553-b4ce-4e4d-92f1-f49f22ef000c service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:15.835219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 727/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:17.786846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8e68-b2ad-41b9-b15c-a78412b532ed None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:17.787380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8e68-b2ad-41b9-b15c-a78412b532ed None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:17.787380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f8e68-b2ad-41b9-b15c-a78412b532ed None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:17.789229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:17.794426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1360e85-26f9-4e9e-bf3b-5faa5c8e7ff5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:17.794939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1360e85-26f9-4e9e-bf3b-5faa5c8e7ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:17.795152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1360e85-26f9-4e9e-bf3b-5faa5c8e7ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:17.859220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e1360e85-26f9-4e9e-bf3b-5faa5c8e7ff5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:17.860403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e1360e85-26f9-4e9e-bf3b-5faa5c8e7ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:17.900838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:17.956252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:18.236166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f8e68-b2ad-41b9-b15c-a78412b532ed None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:18.237714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-617f8e68-b2ad-41b9-b15c-a78412b532ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:18.339875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f8e68-b2ad-41b9-b15c-a78412b532ed None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:18.376461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:18.412035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc23c99-0713-4869-982c-f5eef28f5111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:18.412855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23c99-0713-4869-982c-f5eef28f5111 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:18.413130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23c99-0713-4869-982c-f5eef28f5111 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:18.413395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23c99-0713-4869-982c-f5eef28f5111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:18.415010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bdc23c99-0713-4869-982c-f5eef28f5111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:18.522519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc23c99-0713-4869-982c-f5eef28f5111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:18.523976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc23c99-0713-4869-982c-f5eef28f5111 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:18.541845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 736/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:20.465425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d59a1-c0df-437e-b00a-10e370e0d843 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:20.466023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d59a1-c0df-437e-b00a-10e370e0d843 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:20.466023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d59a1-c0df-437e-b00a-10e370e0d843 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:20.467340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:20.472545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41287999-c71a-48ca-95ea-d4e41d869caf None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:20.472923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41287999-c71a-48ca-95ea-d4e41d869caf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:20.473151 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41287999-c71a-48ca-95ea-d4e41d869caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:20.540939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-41287999-c71a-48ca-95ea-d4e41d869caf None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:20.542145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-41287999-c71a-48ca-95ea-d4e41d869caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:20.560183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:20.635943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:20.917502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d59a1-c0df-437e-b00a-10e370e0d843 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:20.919962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9b8d59a1-c0df-437e-b00a-10e370e0d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:21.018942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d59a1-c0df-437e-b00a-10e370e0d843 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:21.083641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:21.090612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abcf21-916e-43ca-ad50-61e2af55348c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:21.091266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abcf21-916e-43ca-ad50-61e2af55348c service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:21.091518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abcf21-916e-43ca-ad50-61e2af55348c service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:21.091784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abcf21-916e-43ca-ad50-61e2af55348c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:21.093379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d7abcf21-916e-43ca-ad50-61e2af55348c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:21.201703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abcf21-916e-43ca-ad50-61e2af55348c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:21.203278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abcf21-916e-43ca-ad50-61e2af55348c service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:21.221184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 730/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:23.156038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cd9f-f3f2-4def-a9c5-46c9f104dd48 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:23.156698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cd9f-f3f2-4def-a9c5-46c9f104dd48 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:23.156698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3cd9f-f3f2-4def-a9c5-46c9f104dd48 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:23.157922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:23.163813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783b1b-38b0-49b1-8f17-885d9ab12d62 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:23.164224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783b1b-38b0-49b1-8f17-885d9ab12d62 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:23.164368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783b1b-38b0-49b1-8f17-885d9ab12d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:23.231467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a7783b1b-38b0-49b1-8f17-885d9ab12d62 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:23.232692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a7783b1b-38b0-49b1-8f17-885d9ab12d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:23.280897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:23.329032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:23.618226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3cd9f-f3f2-4def-a9c5-46c9f104dd48 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:23.619807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a8c3cd9f-f3f2-4def-a9c5-46c9f104dd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:23.725430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3cd9f-f3f2-4def-a9c5-46c9f104dd48 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:23.773351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:23.799686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df783073-f8e3-4cca-8d96-23818911d0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:23.800312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df783073-f8e3-4cca-8d96-23818911d0cf service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:23.800585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df783073-f8e3-4cca-8d96-23818911d0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:23.800846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df783073-f8e3-4cca-8d96-23818911d0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:23.802814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-df783073-f8e3-4cca-8d96-23818911d0cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:23.913880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df783073-f8e3-4cca-8d96-23818911d0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:23.915403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df783073-f8e3-4cca-8d96-23818911d0cf service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:23.934211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 739/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:33.327194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=22 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:33.931856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=66 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:43.492753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160fbff-51cb-4c8c-86dc-c0a1d8f2f71b None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.493270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160fbff-51cb-4c8c-86dc-c0a1d8f2f71b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.493270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160fbff-51cb-4c8c-86dc-c0a1d8f2f71b None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.494638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 732/1471] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:43.499931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67c82d-d622-49a1-92c1-9d6b992463f1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.500410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67c82d-d622-49a1-92c1-9d6b992463f1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.500515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67c82d-d622-49a1-92c1-9d6b992463f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.564987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3e67c82d-d622-49a1-92c1-9d6b992463f1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:43.566182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3e67c82d-d622-49a1-92c1-9d6b992463f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:43.659703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 740/1472] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:43.955440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de24c7f-318e-4912-a3da-7bc66b5d24df None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.955724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de24c7f-318e-4912-a3da-7bc66b5d24df None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.956001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de24c7f-318e-4912-a3da-7bc66b5d24df None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.957494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 733/1473] 10.222.0.45 () {54 vars in 967 bytes} [Wed Aug 24 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:43.962475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc469b7-fdb1-49b1-9f54-43d8213da87f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.962788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc469b7-fdb1-49b1-9f54-43d8213da87f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.963063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc469b7-fdb1-49b1-9f54-43d8213da87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:44.038344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4bc469b7-fdb1-49b1-9f54-43d8213da87f None None] MFA Rules not processed for user `2f33e1c573c044f086f99eb128c8f726`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:44.039572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4bc469b7-fdb1-49b1-9f54-43d8213da87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:44.060502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:44.119746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 741/1474] 10.222.0.45 () {58 vars in 1064 bytes} [Wed Aug 24 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:44.127363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de24c7f-318e-4912-a3da-7bc66b5d24df None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:44.128795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7de24c7f-318e-4912-a3da-7bc66b5d24df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:44.231421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de24c7f-318e-4912-a3da-7bc66b5d24df None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:44.303020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30368093-8fbb-4216-8e24-b6a219a864d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f33e1c573c044f086f99eb128c8f726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:44.303620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368093-8fbb-4216-8e24-b6a219a864d4 service glance] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:44.303900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368093-8fbb-4216-8e24-b6a219a864d4 service glance] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:44.304176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30368093-8fbb-4216-8e24-b6a219a864d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:44.305766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-30368093-8fbb-4216-8e24-b6a219a864d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:44.358499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:44.412876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30368093-8fbb-4216-8e24-b6a219a864d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:44.414368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30368093-8fbb-4216-8e24-b6a219a864d4 service glance] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:44.433954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 734/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:24:46.425197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4dff-290f-4a17-a714-3bec16beb83d None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:46.425749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4dff-290f-4a17-a714-3bec16beb83d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:46.425749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4dff-290f-4a17-a714-3bec16beb83d None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:46.428058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 742/1476] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:46.433130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829337f-040f-4fd1-9954-cf9722e55759 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:46.433511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829337f-040f-4fd1-9954-cf9722e55759 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:46.433671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829337f-040f-4fd1-9954-cf9722e55759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:46.498321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b829337f-040f-4fd1-9954-cf9722e55759 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:46.499552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b829337f-040f-4fd1-9954-cf9722e55759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:46.593174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 735/1477] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:46.864455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a4dff-290f-4a17-a714-3bec16beb83d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:46.866313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5c1a4dff-290f-4a17-a714-3bec16beb83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:46.990094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a4dff-290f-4a17-a714-3bec16beb83d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:47.038277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:47.081980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67739a74-aa4e-49fd-9e82-95f5b8cbff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:47.082602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739a74-aa4e-49fd-9e82-95f5b8cbff22 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:47.082912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739a74-aa4e-49fd-9e82-95f5b8cbff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:47.083199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739a74-aa4e-49fd-9e82-95f5b8cbff22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:47.083872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67739a74-aa4e-49fd-9e82-95f5b8cbff22 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:47.085598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-67739a74-aa4e-49fd-9e82-95f5b8cbff22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:24:47.085598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:24:47.085598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:24:47.086760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67739a74-aa4e-49fd-9e82-95f5b8cbff22 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:47.104928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 743/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 12:24:46 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:24:49.002078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec9f91-a8f9-439c-9974-37ce5c48a264 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:49.002631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec9f91-a8f9-439c-9974-37ce5c48a264 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:49.002631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec9f91-a8f9-439c-9974-37ce5c48a264 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:49.004077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 736/1479] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:49.009944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd946c01-0652-44b6-a426-bf1370e8cc44 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:49.010384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd946c01-0652-44b6-a426-bf1370e8cc44 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:49.010490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd946c01-0652-44b6-a426-bf1370e8cc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:49.082090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-cd946c01-0652-44b6-a426-bf1370e8cc44 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:49.083287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cd946c01-0652-44b6-a426-bf1370e8cc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:49.179319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 744/1480] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:49.473270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec9f91-a8f9-439c-9974-37ce5c48a264 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:49.474867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b0ec9f91-a8f9-439c-9974-37ce5c48a264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:49.484570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:49.621076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec9f91-a8f9-439c-9974-37ce5c48a264 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:49.698212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:49.729903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ffd77-c258-496c-95f7-7313ed552573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:49.732074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ffd77-c258-496c-95f7-7313ed552573 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:49.732074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ffd77-c258-496c-95f7-7313ed552573 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:49.732074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ffd77-c258-496c-95f7-7313ed552573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:49.732656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-700ffd77-c258-496c-95f7-7313ed552573 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:49.820325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ffd77-c258-496c-95f7-7313ed552573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:49.821924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ffd77-c258-496c-95f7-7313ed552573 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:49.840060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 737/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:51.904097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99912222-3741-450f-b323-84eee2a6db59 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:51.904661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99912222-3741-450f-b323-84eee2a6db59 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:51.904661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99912222-3741-450f-b323-84eee2a6db59 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:51.905954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 745/1482] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:51.911100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368651b3-1d4b-4127-82b6-1be5fe377356 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:51.912043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368651b3-1d4b-4127-82b6-1be5fe377356 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:51.912227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368651b3-1d4b-4127-82b6-1be5fe377356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:51.976998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-368651b3-1d4b-4127-82b6-1be5fe377356 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:51.978202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-368651b3-1d4b-4127-82b6-1be5fe377356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:52.023834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:52.072005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 738/1483] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:52.357414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99912222-3741-450f-b323-84eee2a6db59 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:52.365401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-99912222-3741-450f-b323-84eee2a6db59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:52.384193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:52.462103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99912222-3741-450f-b323-84eee2a6db59 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:52.534553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dafe3-6323-4774-b394-8e789c466145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:52.535175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dafe3-6323-4774-b394-8e789c466145 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:52.535455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dafe3-6323-4774-b394-8e789c466145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:52.535725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dafe3-6323-4774-b394-8e789c466145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:52.537337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ff0dafe3-6323-4774-b394-8e789c466145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:52.609931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:52.644553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0dafe3-6323-4774-b394-8e789c466145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:52.646067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0dafe3-6323-4774-b394-8e789c466145 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:52.663685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 746/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:54.662978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03310-908f-46fe-b577-212ee0ac052c None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:54.663558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03310-908f-46fe-b577-212ee0ac052c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:54.663558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03310-908f-46fe-b577-212ee0ac052c None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:54.664855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 739/1485] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:54.670011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1a09-c6ff-4064-b859-04b3c7c9fd46 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:54.670474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1a09-c6ff-4064-b859-04b3c7c9fd46 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:54.670582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1a09-c6ff-4064-b859-04b3c7c9fd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:54.735488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d24a1a09-c6ff-4064-b859-04b3c7c9fd46 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:54.736706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d24a1a09-c6ff-4064-b859-04b3c7c9fd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:54.832150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 747/1486] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:55.121035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e03310-908f-46fe-b577-212ee0ac052c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:55.122556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d7e03310-908f-46fe-b577-212ee0ac052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:55.222354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e03310-908f-46fe-b577-212ee0ac052c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:55.293668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:55.295423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1121f53-0d02-4be7-9ab2-1b71079e0813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:55.296019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1121f53-0d02-4be7-9ab2-1b71079e0813 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:55.296297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1121f53-0d02-4be7-9ab2-1b71079e0813 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:55.296564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1121f53-0d02-4be7-9ab2-1b71079e0813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:55.298162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a1121f53-0d02-4be7-9ab2-1b71079e0813 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:55.525266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1121f53-0d02-4be7-9ab2-1b71079e0813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:55.526707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1121f53-0d02-4be7-9ab2-1b71079e0813 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:55.544937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 740/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:57.581513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b2937-2879-496e-8cf4-62c802ce790f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:57.582073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b2937-2879-496e-8cf4-62c802ce790f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:57.582073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b2937-2879-496e-8cf4-62c802ce790f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:57.583434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 748/1488] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:57.588726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319938a0-933a-446e-9d01-e44e6fde59e2 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:57.589141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319938a0-933a-446e-9d01-e44e6fde59e2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:57.589267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319938a0-933a-446e-9d01-e44e6fde59e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:57.653406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-319938a0-933a-446e-9d01-e44e6fde59e2 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:57.654558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-319938a0-933a-446e-9d01-e44e6fde59e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:57.707516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:57.750475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 741/1489] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:58.044509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b2937-2879-496e-8cf4-62c802ce790f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:58.046034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1e3b2937-2879-496e-8cf4-62c802ce790f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:58.081543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:58.147886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b2937-2879-496e-8cf4-62c802ce790f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:58.218734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc409a50-0bff-4bf5-8b5d-9454d746246a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:58.219329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc409a50-0bff-4bf5-8b5d-9454d746246a service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:58.219607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc409a50-0bff-4bf5-8b5d-9454d746246a service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:58.219881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc409a50-0bff-4bf5-8b5d-9454d746246a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:58.221455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dc409a50-0bff-4bf5-8b5d-9454d746246a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:58.300132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:58.329961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc409a50-0bff-4bf5-8b5d-9454d746246a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:58.331562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc409a50-0bff-4bf5-8b5d-9454d746246a service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:58.350980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 749/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:00.668691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0d655-4a31-44b0-bbbb-5d1e1ee24f6d None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:00.669266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0d655-4a31-44b0-bbbb-5d1e1ee24f6d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:00.669266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0d655-4a31-44b0-bbbb-5d1e1ee24f6d None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:00.670576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 742/1491] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 12:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:00.675787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d516f-8176-4ea8-ba40-a2b065090bf3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:00.676226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d516f-8176-4ea8-ba40-a2b065090bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:00.676334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d516f-8176-4ea8-ba40-a2b065090bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:00.742712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c16d516f-8176-4ea8-ba40-a2b065090bf3 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:00.743922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c16d516f-8176-4ea8-ba40-a2b065090bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:00.837973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 750/1492] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:01.126698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0d655-4a31-44b0-bbbb-5d1e1ee24f6d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:01.128217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-96f0d655-4a31-44b0-bbbb-5d1e1ee24f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:01.231602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0d655-4a31-44b0-bbbb-5d1e1ee24f6d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:01.306103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e07114-caa8-4707-bc08-96ae358416c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:01.306716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e07114-caa8-4707-bc08-96ae358416c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:01.306998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e07114-caa8-4707-bc08-96ae358416c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:01.307263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e07114-caa8-4707-bc08-96ae358416c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:01.308828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-86e07114-caa8-4707-bc08-96ae358416c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:01.318978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:01.421483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e07114-caa8-4707-bc08-96ae358416c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:01.423331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e07114-caa8-4707-bc08-96ae358416c5 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:01.443430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 743/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:10.836139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=82 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:11.440469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=50 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:22.597659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:22.598779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:22.598898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:22.673681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:22.674918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:22.770440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 751/1494] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:22.778730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b00ffa-962f-4bd1-b90c-1eac63a50e0b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:22.779223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b00ffa-962f-4bd1-b90c-1eac63a50e0b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:22.779328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b00ffa-962f-4bd1-b90c-1eac63a50e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:22.853830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c6b00ffa-962f-4bd1-b90c-1eac63a50e0b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:22.855077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c6b00ffa-962f-4bd1-b90c-1eac63a50e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:22.949365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 744/1495] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:22.954197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:22.955434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:22.967305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24803f-f94f-4fcf-abeb-8d7fa753ff79 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:23.027646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:23.060577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:23.061179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.061755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.062027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.062682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:26:23.062991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:23.063997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:23.063997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:26:23.063997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:26:23.064879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:23.076427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 752/1496] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:26:23.084765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b00ffa-962f-4bd1-b90c-1eac63a50e0b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:23.093794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b00ffa-962f-4bd1-b90c-1eac63a50e0b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:23.154408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:23.187668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:23.188287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.188562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.188830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.189443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:23.192199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING py.warnings [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:23.192199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: warnings.warn(msg) Aug 24 12:26:23.192199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: Aug 24 12:26:23.193542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:23.313412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 745/1497] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:26:23 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:26:23.317552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:23.319334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:23.353976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa0b9b-8a52-4df0-8db1-bb358fcaef19 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:23.445954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:23.446552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.446857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.447128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.447739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:23.448751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:23.448751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:26:23.448751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:26:23.449944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:23.558071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 753/1498] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:26:23 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:26:23.562187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:23.563761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:23.623835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9781b3-736f-41a4-9182-0efe40c82ef9 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:23.710766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:23.720766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:23.721907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.721907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.722114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.722557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:23.723594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING py.warnings [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:23.723594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: warnings.warn(msg) Aug 24 12:26:23.723594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: Aug 24 12:26:23.724455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:23.737065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 746/1499] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:26:23.741588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:23.743119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:23.755588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98b26-8001-408c-b359-4df9847e6833 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:23.783199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:23.851590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c21a20-d7a0-476a-b7a1-d70db847ad1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:23.852116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21a20-d7a0-476a-b7a1-d70db847ad1e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.852392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21a20-d7a0-476a-b7a1-d70db847ad1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.852656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c21a20-d7a0-476a-b7a1-d70db847ad1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.853285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c21a20-d7a0-476a-b7a1-d70db847ad1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:23.854281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-69c21a20-d7a0-476a-b7a1-d70db847ad1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:23.854281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:26:23.854281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:26:23.855192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c21a20-d7a0-476a-b7a1-d70db847ad1e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:23.868703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 754/1500] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:26:23.873138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:23.874667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:23.908221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568a2d-cc5f-4813-a0bf-47b2c97a767c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:24.001408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e813de-1fcb-41fb-88fd-652f37d14ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:24.002138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e813de-1fcb-41fb-88fd-652f37d14ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.002408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e813de-1fcb-41fb-88fd-652f37d14ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.002699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e813de-1fcb-41fb-88fd-652f37d14ecc admin admin] PATH_INFO: `/v3/projects/1fa7b870a4044630a0983e4fd001bb4a/users/7cf66e42e4e64328a105af349c3a6cff/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.024688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e813de-1fcb-41fb-88fd-652f37d14ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa7b870a4044630a0983e4fd001bb4a, user_id=7cf66e42e4e64328a105af349c3a6cff, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:24.025717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING py.warnings [None req-c8e813de-1fcb-41fb-88fd-652f37d14ecc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:24.025717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: warnings.warn(msg) Aug 24 12:26:24.025717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: Aug 24 12:26:24.027654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e813de-1fcb-41fb-88fd-652f37d14ecc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:24.095236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 747/1501] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:26:23 2022] PUT /identity/v3/projects/1fa7b870a4044630a0983e4fd001bb4a/users/7cf66e42e4e64328a105af349c3a6cff/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:26:24.107588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cbb4b-002f-4cdc-9b21-11c3a3e60db8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.107817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cbb4b-002f-4cdc-9b21-11c3a3e60db8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.108092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cbb4b-002f-4cdc-9b21-11c3a3e60db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.166755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-510cbb4b-002f-4cdc-9b21-11c3a3e60db8 None None] MFA Rules not processed for user `7cf66e42e4e64328a105af349c3a6cff`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:24.167970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-510cbb4b-002f-4cdc-9b21-11c3a3e60db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:24.244504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 755/1502] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:24.257840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604f95c-2813-4944-94be-9335d4317eeb None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.258123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604f95c-2813-4944-94be-9335d4317eeb None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.258394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604f95c-2813-4944-94be-9335d4317eeb None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.259760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 748/1503] 10.222.0.45 () {54 vars in 964 bytes} [Wed Aug 24 12:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:24.264522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56cf8-23af-4e88-adfa-5382167aa2ff None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.264808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56cf8-23af-4e88-adfa-5382167aa2ff None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.265084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56cf8-23af-4e88-adfa-5382167aa2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.295918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:24.341074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7cb56cf8-23af-4e88-adfa-5382167aa2ff None None] MFA Rules not processed for user `422f81200e5645668e123229a6f356d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:24.342270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7cb56cf8-23af-4e88-adfa-5382167aa2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:24.416129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 756/1504] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Aug 24 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:24.424114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604f95c-2813-4944-94be-9335d4317eeb None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:24.425747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b604f95c-2813-4944-94be-9335d4317eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:24.474969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:24.528179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604f95c-2813-4944-94be-9335d4317eeb None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:24.599986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:24.600581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.600860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.601133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.602724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:24.698162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:24.699541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:24.702964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:24.717276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 749/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:24.762286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a9cb-1a94-48b9-a871-f76dd5827a1f None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.762726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a9cb-1a94-48b9-a871-f76dd5827a1f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.762862 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a9cb-1a94-48b9-a871-f76dd5827a1f None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.764137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 757/1506] 10.222.0.45 () {52 vars in 830 bytes} [Wed Aug 24 12:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:24.767774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:24.768965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:24.866124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e85dc-1932-455d-9dad-3be7669b80c4 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:24.957025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:24.958215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.958406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.958690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] PATH_INFO: `/v3/users/7cf66e42e4e64328a105af349c3a6cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.970433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf66e42e4e64328a105af349c3a6cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:24.971482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING py.warnings [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:24.971482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: warnings.warn(msg) Aug 24 12:26:24.971482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: Aug 24 12:26:24.972707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:24.981990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:25.186401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf66e42e4e64328a105af349c3a6cff'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:25.186861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf66e42e4e64328a105af349c3a6cff'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:25.187379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf66e42e4e64328a105af349c3a6cff'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:25.201385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf66e42e4e64328a105af349c3a6cff'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:25.223609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf66e42e4e64328a105af349c3a6cff'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:25.293582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 750/1507] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:26:24 2022] DELETE /identity/v3/users/7cf66e42e4e64328a105af349c3a6cff => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:26:25.298071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:25.298382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:25.298674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:25.354836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:25.356030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:25.431479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:25.444469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 758/1508] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:25.454076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:25.455313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:25.554060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ef8ac-098d-4882-974d-c9f127e028c2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:25.607102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:25.629658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c895b-b770-4fca-bf93-4fb66d4a1096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:25.630495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c895b-b770-4fca-bf93-4fb66d4a1096 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:25.630901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c895b-b770-4fca-bf93-4fb66d4a1096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:25.631198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c895b-b770-4fca-bf93-4fb66d4a1096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:25.633105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f45c895b-b770-4fca-bf93-4fb66d4a1096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:25.741411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c895b-b770-4fca-bf93-4fb66d4a1096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:25.742860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c895b-b770-4fca-bf93-4fb66d4a1096 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:25.760595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 751/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:26.263637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:26.264409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:26.275845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f859b0b-c031-4cb5-aac1-786be2c5120d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:26.374190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:26.374848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:26.375125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:26.375455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] PATH_INFO: `/v3/projects/1fa7b870a4044630a0983e4fd001bb4a` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:26.385733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa7b870a4044630a0983e4fd001bb4a)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:26.386790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:26:26.386790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:26:26.386790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:26:26.387974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:26.496082 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa7b870a4044630a0983e4fd001bb4a', 'request_id': 'req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:26.501922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa7b870a4044630a0983e4fd001bb4a', 'request_id': 'req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:26.515153 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa7b870a4044630a0983e4fd001bb4a', 'request_id': 'req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:26.580306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] The token cache is being invalidate because project 1fa7b870a4044630a0983e4fd001bb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:26:26.580592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:26.581015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-00de30fd-7c7b-4149-bf8d-ab1f180d05a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:26:26.582435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 759/1510] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:26:26 2022] DELETE /identity/v3/projects/1fa7b870a4044630a0983e4fd001bb4a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:26:35.758380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=62 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:36.578197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:54.884260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405abddc-19f5-45e3-94e7-4a66813ff6b7 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:54.884858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405abddc-19f5-45e3-94e7-4a66813ff6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:54.884858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405abddc-19f5-45e3-94e7-4a66813ff6b7 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:54.886150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:54.890859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028831c-6448-4e32-bd4d-63a8d6acdf41 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:54.891294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028831c-6448-4e32-bd4d-63a8d6acdf41 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:54.891399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028831c-6448-4e32-bd4d-63a8d6acdf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:54.975937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d028831c-6448-4e32-bd4d-63a8d6acdf41 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:54.977134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d028831c-6448-4e32-bd4d-63a8d6acdf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:55.078482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:55.233052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405abddc-19f5-45e3-94e7-4a66813ff6b7 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:55.235208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-405abddc-19f5-45e3-94e7-4a66813ff6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:55.351272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405abddc-19f5-45e3-94e7-4a66813ff6b7 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:55.356656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de0414-d697-46bc-871a-8732151bfffc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:55.357249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de0414-d697-46bc-871a-8732151bfffc service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:55.357524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de0414-d697-46bc-871a-8732151bfffc service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:55.357791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de0414-d697-46bc-871a-8732151bfffc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:55.366480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de0414-d697-46bc-871a-8732151bfffc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:55.368069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de0414-d697-46bc-871a-8732151bfffc service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:55.377161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 753/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:56.201719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f9c39-18c9-4916-9b8a-ba9df858ec14 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:56.202223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f9c39-18c9-4916-9b8a-ba9df858ec14 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:56.202344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f9c39-18c9-4916-9b8a-ba9df858ec14 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:56.203630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:56.208498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4351-c7f7-4852-b496-51286b5fd59f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:56.208943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4351-c7f7-4852-b496-51286b5fd59f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:56.209055 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a4351-c7f7-4852-b496-51286b5fd59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:56.229725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0b4a4351-c7f7-4852-b496-51286b5fd59f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:56.230972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0b4a4351-c7f7-4852-b496-51286b5fd59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:56.238686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:56.390003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f9c39-18c9-4916-9b8a-ba9df858ec14 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:56.404550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f9c39-18c9-4916-9b8a-ba9df858ec14 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:56.409806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64d383-e16e-4fcd-947d-2f3b881de7c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:56.410373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64d383-e16e-4fcd-947d-2f3b881de7c8 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:56.410679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64d383-e16e-4fcd-947d-2f3b881de7c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:56.410947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64d383-e16e-4fcd-947d-2f3b881de7c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:56.418941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64d383-e16e-4fcd-947d-2f3b881de7c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:56.420513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64d383-e16e-4fcd-947d-2f3b881de7c8 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:56.430764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 762/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:57.123434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa062e1c-5bc9-481c-82b9-81bce944375c None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:57.123714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa062e1c-5bc9-481c-82b9-81bce944375c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:57.123985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa062e1c-5bc9-481c-82b9-81bce944375c None None] PATH_INFO: `/v3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:57.125304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 755/1517] 10.222.0.60 () {54 vars in 925 bytes} [Wed Aug 24 12:27:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:27:57.132035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfca8c-63d7-454b-9c86-82a5fde5f9dd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:57.132319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfca8c-63d7-454b-9c86-82a5fde5f9dd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:57.132642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfca8c-63d7-454b-9c86-82a5fde5f9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:57.191319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-15dfca8c-63d7-454b-9c86-82a5fde5f9dd None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:57.192529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-15dfca8c-63d7-454b-9c86-82a5fde5f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:57.208606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:57.262760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 763/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 24 12:27:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:06.224964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=56 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:28:06.521506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0355099-d720-4cb7-b9f6-3d5284782290 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:06.521792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0355099-d720-4cb7-b9f6-3d5284782290 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:06.522069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0355099-d720-4cb7-b9f6-3d5284782290 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:06.523443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 756/1519] 10.222.0.45 () {54 vars in 961 bytes} [Wed Aug 24 12:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:28:06.528848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758510af-8d80-4c59-8b69-b38bebdf6206 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:06.529342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758510af-8d80-4c59-8b69-b38bebdf6206 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:06.529452 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758510af-8d80-4c59-8b69-b38bebdf6206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:06.548371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-758510af-8d80-4c59-8b69-b38bebdf6206 None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:06.549572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-758510af-8d80-4c59-8b69-b38bebdf6206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:06.558116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 764/1520] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Aug 24 12:28:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:06.565333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0355099-d720-4cb7-b9f6-3d5284782290 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:28:06.578873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0355099-d720-4cb7-b9f6-3d5284782290 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:28:06.583896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c564e-7859-4dea-8350-ecd1ca90225c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:28:06.584470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c564e-7859-4dea-8350-ecd1ca90225c service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:06.584745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c564e-7859-4dea-8350-ecd1ca90225c service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:06.585010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c564e-7859-4dea-8350-ecd1ca90225c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:06.595859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c564e-7859-4dea-8350-ecd1ca90225c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:28:06.597438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c564e-7859-4dea-8350-ecd1ca90225c service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:28:06.607078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 757/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:28:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:28:07.655665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f84f3-60e3-4c0b-b05f-a015f29ebec5 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:07.656181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f84f3-60e3-4c0b-b05f-a015f29ebec5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:07.656181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f84f3-60e3-4c0b-b05f-a015f29ebec5 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:07.657733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:28:07.662480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b70679-e700-4020-918c-3613db31eb1e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:07.668152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b70679-e700-4020-918c-3613db31eb1e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:07.668152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b70679-e700-4020-918c-3613db31eb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:07.682570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f4b70679-e700-4020-918c-3613db31eb1e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:07.683798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f4b70679-e700-4020-918c-3613db31eb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:07.691641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:07.846399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f84f3-60e3-4c0b-b05f-a015f29ebec5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:28:07.861075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f84f3-60e3-4c0b-b05f-a015f29ebec5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:28:07.866684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31284c29-afd4-43b6-bbb3-94dfc069ae79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:28:07.867267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31284c29-afd4-43b6-bbb3-94dfc069ae79 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:07.867549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31284c29-afd4-43b6-bbb3-94dfc069ae79 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:07.867820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31284c29-afd4-43b6-bbb3-94dfc069ae79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:07.876042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31284c29-afd4-43b6-bbb3-94dfc069ae79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:28:07.877653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31284c29-afd4-43b6-bbb3-94dfc069ae79 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:28:07.886990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 766/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:28:08.705168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0a3e8-82c5-4ef9-9abd-01052df3be22 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:08.705674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0a3e8-82c5-4ef9-9abd-01052df3be22 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:08.705674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0a3e8-82c5-4ef9-9abd-01052df3be22 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:08.707030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:28:08.711766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d99d6-c19f-4146-a33d-7de479df40fc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:08.712193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d99d6-c19f-4146-a33d-7de479df40fc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:08.712304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d99d6-c19f-4146-a33d-7de479df40fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:08.732317 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ea5d99d6-c19f-4146-a33d-7de479df40fc None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:08.733502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ea5d99d6-c19f-4146-a33d-7de479df40fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:08.741387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:08.894456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0a3e8-82c5-4ef9-9abd-01052df3be22 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:28:08.908577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0a3e8-82c5-4ef9-9abd-01052df3be22 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:28:08.913922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e331a-7982-40bf-8614-f78835b17f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:28:08.914505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e331a-7982-40bf-8614-f78835b17f48 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:08.915323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e331a-7982-40bf-8614-f78835b17f48 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:08.915589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e331a-7982-40bf-8614-f78835b17f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:08.923605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e331a-7982-40bf-8614-f78835b17f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:28:08.925227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e331a-7982-40bf-8614-f78835b17f48 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:28:08.935171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 760/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:28:12.757410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fb6d8-6a62-46d5-95ed-1683b712a4aa None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:12.757969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fb6d8-6a62-46d5-95ed-1683b712a4aa None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:12.758068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fb6d8-6a62-46d5-95ed-1683b712a4aa None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:12.759397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:28:12.764657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5797a6-ba22-4c5a-b784-4e7f87271063 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:12.766780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5797a6-ba22-4c5a-b784-4e7f87271063 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:12.766780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5797a6-ba22-4c5a-b784-4e7f87271063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:12.784740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7f5797a6-ba22-4c5a-b784-4e7f87271063 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:12.785925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7f5797a6-ba22-4c5a-b784-4e7f87271063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:12.794190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:13.077473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fb6d8-6a62-46d5-95ed-1683b712a4aa None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:28:13.079200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-144fb6d8-6a62-46d5-95ed-1683b712a4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:13.155722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fb6d8-6a62-46d5-95ed-1683b712a4aa None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:28:13.159523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28769f15-cbf7-4516-a2fa-5bef9f57d6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:28:13.160129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28769f15-cbf7-4516-a2fa-5bef9f57d6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:13.160412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28769f15-cbf7-4516-a2fa-5bef9f57d6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:13.160686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28769f15-cbf7-4516-a2fa-5bef9f57d6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:13.170231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28769f15-cbf7-4516-a2fa-5bef9f57d6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:28:13.171685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28769f15-cbf7-4516-a2fa-5bef9f57d6a8 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:28:13.180839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 769/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:28:15.050324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396141c-f1e3-47ef-9f06-217b9fa8c792 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:15.050936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396141c-f1e3-47ef-9f06-217b9fa8c792 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:15.050936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396141c-f1e3-47ef-9f06-217b9fa8c792 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:15.052270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:28:15.057848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953d17-8adb-4de3-bfdf-d9954677db6a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:15.058312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953d17-8adb-4de3-bfdf-d9954677db6a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:15.058421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd953d17-8adb-4de3-bfdf-d9954677db6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:15.078998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fd953d17-8adb-4de3-bfdf-d9954677db6a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:15.080172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fd953d17-8adb-4de3-bfdf-d9954677db6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:15.088447 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:15.369691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396141c-f1e3-47ef-9f06-217b9fa8c792 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:28:15.380436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396141c-f1e3-47ef-9f06-217b9fa8c792 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:28:15.384412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dc2ee-5f09-48fc-8f04-af54a4f2e606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:28:15.384987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dc2ee-5f09-48fc-8f04-af54a4f2e606 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:15.385262 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dc2ee-5f09-48fc-8f04-af54a4f2e606 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:15.385529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dc2ee-5f09-48fc-8f04-af54a4f2e606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:15.394056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dc2ee-5f09-48fc-8f04-af54a4f2e606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:28:15.395831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dc2ee-5f09-48fc-8f04-af54a4f2e606 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:28:15.405132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 763/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:28:18.810404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e99508-ff66-4fea-b2dc-b4389764c72a None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:18.810963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e99508-ff66-4fea-b2dc-b4389764c72a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:18.810963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e99508-ff66-4fea-b2dc-b4389764c72a None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:18.812411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:28:18.817955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27bd46-74c0-4d5a-a9d9-b820ca04d9cd None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:18.818393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27bd46-74c0-4d5a-a9d9-b820ca04d9cd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:18.818537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27bd46-74c0-4d5a-a9d9-b820ca04d9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:18.839515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-bf27bd46-74c0-4d5a-a9d9-b820ca04d9cd None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:18.840831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-bf27bd46-74c0-4d5a-a9d9-b820ca04d9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:18.848933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:19.399254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e99508-ff66-4fea-b2dc-b4389764c72a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:28:19.400927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-21e99508-ff66-4fea-b2dc-b4389764c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:19.465838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:28:19.479075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e99508-ff66-4fea-b2dc-b4389764c72a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:28:19.482625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae16fc5-e2c3-48f6-8205-3b031adc03aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:28:19.483241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae16fc5-e2c3-48f6-8205-3b031adc03aa service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:19.483522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae16fc5-e2c3-48f6-8205-3b031adc03aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:19.483790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae16fc5-e2c3-48f6-8205-3b031adc03aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:19.493473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae16fc5-e2c3-48f6-8205-3b031adc03aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:28:19.494956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae16fc5-e2c3-48f6-8205-3b031adc03aa service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:28:19.504652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 772/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:28:21.405704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1429-40ea-4023-b2be-6afb53b0650a None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:21.406229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1429-40ea-4023-b2be-6afb53b0650a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:21.406229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1429-40ea-4023-b2be-6afb53b0650a None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:21.407617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:28:21.412967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952617f4-cf1c-459e-966e-61fe472ddaa2 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:21.413377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952617f4-cf1c-459e-966e-61fe472ddaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:21.413517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952617f4-cf1c-459e-966e-61fe472ddaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:21.433139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-952617f4-cf1c-459e-966e-61fe472ddaa2 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:21.434324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-952617f4-cf1c-459e-966e-61fe472ddaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:21.441995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:21.994907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1429-40ea-4023-b2be-6afb53b0650a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:28:21.996576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fa9a1429-40ea-4023-b2be-6afb53b0650a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:22.009870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1429-40ea-4023-b2be-6afb53b0650a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:28:22.013348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed43b0-cb6f-4b1f-8fe6-10a874060555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:28:22.013926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed43b0-cb6f-4b1f-8fe6-10a874060555 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:22.014205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed43b0-cb6f-4b1f-8fe6-10a874060555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:22.014471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed43b0-cb6f-4b1f-8fe6-10a874060555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:22.023195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed43b0-cb6f-4b1f-8fe6-10a874060555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:28:22.024604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed43b0-cb6f-4b1f-8fe6-10a874060555 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:28:22.033771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 766/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:28:31.432064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=11 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:28:32.029754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=33 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:45.732732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444d03-ae0e-4032-92e0-b41f3227820e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:45.736500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444d03-ae0e-4032-92e0-b41f3227820e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:45.736500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444d03-ae0e-4032-92e0-b41f3227820e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:45.753731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-57444d03-ae0e-4032-92e0-b41f3227820e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:45.759083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-57444d03-ae0e-4032-92e0-b41f3227820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:45.774059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 774/1540] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:45.782561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac004277-5c6c-4b57-84f5-7702ce342c9e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:45.783147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac004277-5c6c-4b57-84f5-7702ce342c9e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:45.783147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac004277-5c6c-4b57-84f5-7702ce342c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:45.812737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ac004277-5c6c-4b57-84f5-7702ce342c9e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:45.813942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ac004277-5c6c-4b57-84f5-7702ce342c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:45.836968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 767/1541] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:45.854560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57444d03-ae0e-4032-92e0-b41f3227820e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:45.865529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57444d03-ae0e-4032-92e0-b41f3227820e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:45.874409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:45.877992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:45.878323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:45.878532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:45.879255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:45.879459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:45.881780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:45.899625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 775/1542] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:45.907952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac004277-5c6c-4b57-84f5-7702ce342c9e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:45.920793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac004277-5c6c-4b57-84f5-7702ce342c9e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:45.929809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b73fb9-bfe8-4383-9337-7d6c804a7c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:45.930450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b73fb9-bfe8-4383-9337-7d6c804a7c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:45.930757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b73fb9-bfe8-4383-9337-7d6c804a7c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:45.931027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b73fb9-bfe8-4383-9337-7d6c804a7c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:45.931663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b73fb9-bfe8-4383-9337-7d6c804a7c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:45.937086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b73fb9-bfe8-4383-9337-7d6c804a7c3d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.096134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 768/1543] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:45 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:46.101019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.112000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15f4c-9a85-4573-aadf-9b8077832cbb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.131712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883279db-1f1d-440e-83f0-c49d487273e9 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.132096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883279db-1f1d-440e-83f0-c49d487273e9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.132287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883279db-1f1d-440e-83f0-c49d487273e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.163498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-883279db-1f1d-440e-83f0-c49d487273e9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.169699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-883279db-1f1d-440e-83f0-c49d487273e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.214072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3578e-a009-4751-a11d-8f45ad54790c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.214708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3578e-a009-4751-a11d-8f45ad54790c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.214991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3578e-a009-4751-a11d-8f45ad54790c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.215262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3578e-a009-4751-a11d-8f45ad54790c admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.215913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3578e-a009-4751-a11d-8f45ad54790c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.217881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3578e-a009-4751-a11d-8f45ad54790c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.306905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 769/1544] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:29:46.314399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ced22-77d8-4e6e-9eb1-279bdc7fe9f9 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.314730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ced22-77d8-4e6e-9eb1-279bdc7fe9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.315014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ced22-77d8-4e6e-9eb1-279bdc7fe9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.342400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 776/1545] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/users => generated 283 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:46.343880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-022ced22-77d8-4e6e-9eb1-279bdc7fe9f9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.349247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-022ced22-77d8-4e6e-9eb1-279bdc7fe9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.349413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebad11d-e80d-4c86-a967-0cbd109e072a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.349413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebad11d-e80d-4c86-a967-0cbd109e072a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.349413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebad11d-e80d-4c86-a967-0cbd109e072a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.364959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 770/1546] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:29:46.365881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ced22-77d8-4e6e-9eb1-279bdc7fe9f9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.378319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ced22-77d8-4e6e-9eb1-279bdc7fe9f9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.390290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2ebad11d-e80d-4c86-a967-0cbd109e072a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.390290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2ebad11d-e80d-4c86-a967-0cbd109e072a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.392692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda5669-a5c6-49fc-a3a9-50e52b8be7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.395562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 777/1547] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.395807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda5669-a5c6-49fc-a3a9-50e52b8be7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.396598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda5669-a5c6-49fc-a3a9-50e52b8be7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.396873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda5669-a5c6-49fc-a3a9-50e52b8be7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.397516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda5669-a5c6-49fc-a3a9-50e52b8be7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.398746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebad11d-e80d-4c86-a967-0cbd109e072a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.400254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda5669-a5c6-49fc-a3a9-50e52b8be7c7 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.414035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebad11d-e80d-4c86-a967-0cbd109e072a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.417366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 771/1548] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:46.418680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b2f3-4680-47f5-82fa-d10bf12cf285 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.424753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b2f3-4680-47f5-82fa-d10bf12cf285 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.424753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b2f3-4680-47f5-82fa-d10bf12cf285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.425146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a2d21-034f-452f-883a-4b2ed515d552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.425766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2d21-034f-452f-883a-4b2ed515d552 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.426040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2d21-034f-452f-883a-4b2ed515d552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.426308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2d21-034f-452f-883a-4b2ed515d552 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.426979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a2d21-034f-452f-883a-4b2ed515d552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:46.427313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a2d21-034f-452f-883a-4b2ed515d552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.428908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a2d21-034f-452f-883a-4b2ed515d552 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.442499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 778/1549] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:46.445023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a5ea3-6d4d-4151-9560-efa090bc7b36 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.445311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a5ea3-6d4d-4151-9560-efa090bc7b36 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.445587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a5ea3-6d4d-4151-9560-efa090bc7b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.473068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-be0a5ea3-6d4d-4151-9560-efa090bc7b36 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.474477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-be0a5ea3-6d4d-4151-9560-efa090bc7b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.474843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-51d1b2f3-4680-47f5-82fa-d10bf12cf285 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.482629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-51d1b2f3-4680-47f5-82fa-d10bf12cf285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.483452 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 779/1550] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.484752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba508-2411-4176-8273-3c5a1a6b25a1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.485036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba508-2411-4176-8273-3c5a1a6b25a1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.485530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ba508-2411-4176-8273-3c5a1a6b25a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.489513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 772/1551] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.491347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b2f3-4680-47f5-82fa-d10bf12cf285 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.513294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b2f3-4680-47f5-82fa-d10bf12cf285 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.518255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.518993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.519280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.519552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.519896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-715ba508-2411-4176-8273-3c5a1a6b25a1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.520200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.521687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-715ba508-2411-4176-8273-3c5a1a6b25a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.522748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.534500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 773/1552] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:46.535541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.538273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 780/1553] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.539220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ba508-2411-4176-8273-3c5a1a6b25a1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.546620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97f7dc-9582-48cb-8cb9-f35fe7659798 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.549330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ba508-2411-4176-8273-3c5a1a6b25a1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.552210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e298b3-978a-41af-ad0a-f9f450c7c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.553210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e298b3-978a-41af-ad0a-f9f450c7c897 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.553469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e298b3-978a-41af-ad0a-f9f450c7c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.553742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e298b3-978a-41af-ad0a-f9f450c7c897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.554385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e298b3-978a-41af-ad0a-f9f450c7c897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.555186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded58ac-a6c4-4923-b9a0-3fededc6f2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.555990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded58ac-a6c4-4923-b9a0-3fededc6f2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.556301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded58ac-a6c4-4923-b9a0-3fededc6f2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.556480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e298b3-978a-41af-ad0a-f9f450c7c897 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.556677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded58ac-a6c4-4923-b9a0-3fededc6f2eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.557214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded58ac-a6c4-4923-b9a0-3fededc6f2eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:46.557538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded58ac-a6c4-4923-b9a0-3fededc6f2eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.559361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded58ac-a6c4-4923-b9a0-3fededc6f2eb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.571851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 781/1554] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:46.573227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8544bf-69ce-449c-ac23-081f91bee4a8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.573573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8544bf-69ce-449c-ac23-081f91bee4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.573907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8544bf-69ce-449c-ac23-081f91bee4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.593126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8e8544bf-69ce-449c-ac23-081f91bee4a8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.594323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8e8544bf-69ce-449c-ac23-081f91bee4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.602897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 782/1555] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.603996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ddd86-d831-4aee-98d2-476bfd642e77 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.604276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ddd86-d831-4aee-98d2-476bfd642e77 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.604550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ddd86-d831-4aee-98d2-476bfd642e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.623780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b26ddd86-d831-4aee-98d2-476bfd642e77 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.624971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b26ddd86-d831-4aee-98d2-476bfd642e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.634700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 783/1556] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.635931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7df2ab-53ef-4b89-866f-e502f84f1252 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.636213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7df2ab-53ef-4b89-866f-e502f84f1252 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.636488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7df2ab-53ef-4b89-866f-e502f84f1252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.655094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ed7df2ab-53ef-4b89-866f-e502f84f1252 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.656312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ed7df2ab-53ef-4b89-866f-e502f84f1252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.664282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 784/1557] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.665020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7df2ab-53ef-4b89-866f-e502f84f1252 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.674406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7df2ab-53ef-4b89-866f-e502f84f1252 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.680241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 774/1558] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:46.680241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c71f23-4e15-49d0-8319-d96e8e9a868e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.680241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c71f23-4e15-49d0-8319-d96e8e9a868e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.680241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c71f23-4e15-49d0-8319-d96e8e9a868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.698978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-58c71f23-4e15-49d0-8319-d96e8e9a868e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.702983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-58c71f23-4e15-49d0-8319-d96e8e9a868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.733348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925112fb-0348-4510-8751-f31a341e3859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.733997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925112fb-0348-4510-8751-f31a341e3859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.734275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925112fb-0348-4510-8751-f31a341e3859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.734543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925112fb-0348-4510-8751-f31a341e3859 admin admin] PATH_INFO: `/v3/projects/d6bc9523696d4a50a8ae80631a03a6ed/users/6a9228ec00ef4ff5b2a90992499f4d8f/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.742703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:46.750313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925112fb-0348-4510-8751-f31a341e3859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc9523696d4a50a8ae80631a03a6ed, user_id=6a9228ec00ef4ff5b2a90992499f4d8f, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.751455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-925112fb-0348-4510-8751-f31a341e3859 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:46.751455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:29:46.751455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:29:46.753362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925112fb-0348-4510-8751-f31a341e3859 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.758951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 775/1559] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.760423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8ec8b-f3fe-4951-8516-ab58e7496d79 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.760588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8ec8b-f3fe-4951-8516-ab58e7496d79 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.760717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8ec8b-f3fe-4951-8516-ab58e7496d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.774127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:46.779977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d4c8ec8b-f3fe-4951-8516-ab58e7496d79 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.781173 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d4c8ec8b-f3fe-4951-8516-ab58e7496d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.789056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 776/1560] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.789832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8ec8b-f3fe-4951-8516-ab58e7496d79 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.798808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8ec8b-f3fe-4951-8516-ab58e7496d79 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.804036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271302d-5288-40ce-8f9e-918e9a595933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.804598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271302d-5288-40ce-8f9e-918e9a595933 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.804875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271302d-5288-40ce-8f9e-918e9a595933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.805143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271302d-5288-40ce-8f9e-918e9a595933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.805751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271302d-5288-40ce-8f9e-918e9a595933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.807671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271302d-5288-40ce-8f9e-918e9a595933 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.817110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 785/1561] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:46 2022] PUT /identity/v3/projects/d6bc9523696d4a50a8ae80631a03a6ed/users/6a9228ec00ef4ff5b2a90992499f4d8f/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:46.818991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48331d5d-8426-43a8-8f35-32f5f113f90d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.819278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48331d5d-8426-43a8-8f35-32f5f113f90d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.819550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48331d5d-8426-43a8-8f35-32f5f113f90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.839477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-48331d5d-8426-43a8-8f35-32f5f113f90d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.840707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-48331d5d-8426-43a8-8f35-32f5f113f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.897930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 786/1562] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.898719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48331d5d-8426-43a8-8f35-32f5f113f90d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.907018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 777/1563] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:46.907018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19bb15-81e2-43d3-b4fb-8b97a4c0df0c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.907018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19bb15-81e2-43d3-b4fb-8b97a4c0df0c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.907483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19bb15-81e2-43d3-b4fb-8b97a4c0df0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.908203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48331d5d-8426-43a8-8f35-32f5f113f90d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.926052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-4b19bb15-81e2-43d3-b4fb-8b97a4c0df0c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:46.927884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4b19bb15-81e2-43d3-b4fb-8b97a4c0df0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:46.967355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:46.967961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.968239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.968511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:46.969159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:46.969457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:46.971108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:46.983023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 787/1564] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:46.983821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:46.995101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 778/1565] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:46.996441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e2d4-301e-46b9-9c25-25626523ab44 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:46.996727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e2d4-301e-46b9-9c25-25626523ab44 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:46.996940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8066b27-764e-4981-967e-bd750503358f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:46.997057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517e2d4-301e-46b9-9c25-25626523ab44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.001484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.002073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.002349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.002619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.003248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.005150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.015954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-4517e2d4-301e-46b9-9c25-25626523ab44 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:47.017219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4517e2d4-301e-46b9-9c25-25626523ab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:47.025115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 779/1566] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:47.027518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517e2d4-301e-46b9-9c25-25626523ab44 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.036372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517e2d4-301e-46b9-9c25-25626523ab44 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.041364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.041936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.042210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.042479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.043164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:47.043462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.044463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING py.warnings [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:47.044463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: warnings.warn(msg) Aug 24 12:29:47.044463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: Aug 24 12:29:47.045329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655878-2ba7-4734-8b57-3655e06bf985 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.057246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 780/1567] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:47.058426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8425b-249b-4a67-b525-5b8ff20f4724 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.058769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8425b-249b-4a67-b525-5b8ff20f4724 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.059047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8425b-249b-4a67-b525-5b8ff20f4724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.082454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 788/1568] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:46 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.082454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.085921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-64a8425b-249b-4a67-b525-5b8ff20f4724 None None] MFA Rules not processed for user `6a9228ec00ef4ff5b2a90992499f4d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:47.087302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-64a8425b-249b-4a67-b525-5b8ff20f4724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:47.090029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44bda-3ef0-4bcf-af79-209d253af738 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.094596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.095196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.095478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.095748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.096411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:47.096713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.098299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.111464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 789/1569] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:47.112086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.122807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd092e-afd4-4583-b840-07c9b84f0f4f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.127632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.128209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.128497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.128770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.129414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.131353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.154547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 781/1570] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:47.155349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8425b-249b-4a67-b525-5b8ff20f4724 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.165487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8425b-249b-4a67-b525-5b8ff20f4724 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.169987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.171010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.171010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.171212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.171697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.173575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.211324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 790/1571] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.212422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.221979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf738e5-63d3-44c3-8a4a-0e353a1e284b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.227106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.227670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.227949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.228220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.228842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:47.229141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.230771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.243878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 791/1572] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:47.244863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.254628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40c0af-7625-4920-a752-e6717a541d4e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.261325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.262346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.262873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.263346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.264443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:47.264940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.266610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.278693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 792/1573] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:47.279614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.289716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4b41f-68a5-4497-8163-3956d97f9c19 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.293895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 782/1574] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.294618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.295408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.296202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.296823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.297212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.297946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.299137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:47.299137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:29:47.299137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:29:47.300329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.305564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:47.325215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:47.356597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b464fb-8b0f-450d-8648-50648bd3859d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.361292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.361887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.362197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.362467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.363460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.366398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.376060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 783/1575] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:47.380953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.386917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296d8b5-7021-451b-8310-5115177fc634 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.391389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.391937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.392216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.392487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.393075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.394996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.412820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 793/1576] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/projects => generated 393 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.412820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.472493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332c218-9635-43b0-8b5d-8ffcb8e00015 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.477112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.477692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.477973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.478245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.478893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.480793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.524221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 784/1577] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.525051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.584063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6148-d259-4210-921b-ee11e2a6ead4 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.588827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.589417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.589689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.589971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.590598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.592274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.596643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 794/1578] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.596643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.605332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 785/1579] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:47.605332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.613847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c92a97-0eaa-4fc9-8432-e9636603d9eb admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.655956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e89f0-7ba9-4da2-a0c8-9a7ada5de9c9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.660590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.661205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.661495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.661773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.662395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.664412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.670141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.670803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.671081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.671354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.671970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.673858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.749207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 795/1580] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.750149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.777828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 786/1581] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.778532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.806511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:47.807624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:47.812722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9180a-eddd-4d45-b160-d4a746dffe20 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.841752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae29d0-1f70-4a8d-b2d4-f0c54f1f5a56 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.846438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.847065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.847345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.847617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.848251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.850503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.860207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 787/1582] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:47.860990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.870198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.870832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.871113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.871214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f21b88-ea08-4141-bd00-7124f3a71f5d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.871383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.872008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.874000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.875854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.876412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.876690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.876961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.877567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.879052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING py.warnings [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:47.879052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: warnings.warn(msg) Aug 24 12:29:47.879052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: Aug 24 12:29:47.880613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:47.980760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 796/1583] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.980760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.988195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7eec18-a80d-4974-bb7b-815bc32f7286 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:47.988486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 788/1584] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:47.989310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:47.996434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:47.997039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:47.997349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:47.997712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:47.998402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:47.999685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a614a5-9b30-4397-b7fb-539e9cfe722a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.000371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.004128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.008625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.008625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.008625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.008625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.008625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.085483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 797/1585] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:48.086511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.095983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e0fd3-551b-45d5-b935-c1379b7ec1bb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.101086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.101651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.101929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.102201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.102854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.104447 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.108249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 789/1586] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:47 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:48.109116 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.115541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 798/1587] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.116385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.121988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee5e46-3273-418f-969b-e480f825be8a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.125969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383ce5b-81d5-4b4d-a4d0-d7b87dbca571 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.126738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.127324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.127600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.127866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.128470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.130457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.130805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.131407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.131671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.131943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.132550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.134429 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.238277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 790/1588] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:48 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:48.241649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.248791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce80e7-0455-4738-bf90-bc09523da031 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.253413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.254017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.254399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.254689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] PATH_INFO: `/v3/projects/94e584086ede4dcfb7fa3002b1091b8e/users/7750a81cb89346ecb3634daa137d0da3/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.269976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 799/1589] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:48 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:48.269976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.270539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e584086ede4dcfb7fa3002b1091b8e, user_id=7750a81cb89346ecb3634daa137d0da3, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.273751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.279746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae80ee-4416-4dd0-abd5-6002bd68990f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.284308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.284886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.285163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.285432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.286096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.288255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.298013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 800/1590] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.298799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.307965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85de2e-c2c9-4db4-b83c-748d917baff0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.312696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.313248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.313523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.313790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.314392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.315973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.327379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 801/1591] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.328141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.337819 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224583f9-1572-45c3-a111-3c5d184ad56b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.342602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.343256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.343439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.343705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.344311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.345926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.348520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 791/1592] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:48 2022] PUT /identity/v3/projects/94e584086ede4dcfb7fa3002b1091b8e/users/7750a81cb89346ecb3634daa137d0da3/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:48.354788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.359188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 802/1593] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.359188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.374779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12498868-b3b7-46cf-84dc-60cda682e07a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.416333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b6119-6465-4792-9bb9-152b0b13198d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.420556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.421350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.421627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.421895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.425486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.425486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.433211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.433866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.437462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.437582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 792/1594] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.437665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] PATH_INFO: `/v3/projects/504b03560cf34426a08958b471c91646/users/6b94f244190e4af089d070ad83806c38/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.437765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.444808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce45604-91c0-46e8-bf28-46622482ea83 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.449443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.450001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.450269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.450533 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.451178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.451456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504b03560cf34426a08958b471c91646, user_id=6b94f244190e4af089d070ad83806c38, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.452728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.454083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.457107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:48.466151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 793/1595] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.466151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.473149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d78a3-3caa-477e-b11f-2ba81b9d5d4b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.477858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.478458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.478756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.479031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.479649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.481238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.491782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 794/1596] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.492501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.501580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755ff5f-2c49-482b-8dc8-0dca1eba0758 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.506612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.507172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.507449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.507713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.508306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.509814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.519207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 795/1597] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.519947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.527309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:48.530998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fcbde-2f08-4655-a1bb-5d5cea0e00f9 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.531417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 803/1598] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:48 2022] PUT /identity/v3/projects/504b03560cf34426a08958b471c91646/users/6b94f244190e4af089d070ad83806c38/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:48.532231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.589493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbb6db-a56c-4246-9fb8-ca7e31ebb6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.590084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb6db-a56c-4246-9fb8-ca7e31ebb6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.590592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb6db-a56c-4246-9fb8-ca7e31ebb6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.590895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbb6db-a56c-4246-9fb8-ca7e31ebb6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.591706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbb6db-a56c-4246-9fb8-ca7e31ebb6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.593772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbb6db-a56c-4246-9fb8-ca7e31ebb6d0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.594766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9a19e-ca76-4ff1-aa61-f99f1bdcc239 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.600156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.600757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.602269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.602269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.602269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.603540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.604003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 796/1599] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.605209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a5ac1-5627-4add-a7f1-62cf6e5698a2 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.605487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a5ac1-5627-4add-a7f1-62cf6e5698a2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.605756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a5ac1-5627-4add-a7f1-62cf6e5698a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.613938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 804/1600] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.615722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.625487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7f93c-24ef-4198-8635-9fb97a22625e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.636912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.636912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.636912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.636912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] PATH_INFO: `/v3/projects/03762335be8c4eb5b1eeb58da678b2b9/users/b3b3ab610ea142689067612f78a221b3/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.637326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-248a5ac1-5627-4add-a7f1-62cf6e5698a2 None None] MFA Rules not processed for user `7750a81cb89346ecb3634daa137d0da3`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:48.637326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-248a5ac1-5627-4add-a7f1-62cf6e5698a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:48.645378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=03762335be8c4eb5b1eeb58da678b2b9, user_id=b3b3ab610ea142689067612f78a221b3, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.648045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.698226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 797/1601] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:48.699090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a5ac1-5627-4add-a7f1-62cf6e5698a2 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.709837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a5ac1-5627-4add-a7f1-62cf6e5698a2 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.714522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.715137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.715416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.715684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.716323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.717901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.727124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 805/1602] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:48 2022] PUT /identity/v3/projects/03762335be8c4eb5b1eeb58da678b2b9/users/b3b3ab610ea142689067612f78a221b3/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:48.728102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.729078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 798/1603] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.730032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.741478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0102a66-d0e4-4edb-8b8e-213fde7776a8 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.796662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da3451-6af0-44a5-a92a-9eef4c0f99fa admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.798255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.798939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.799218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.799486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] PATH_INFO: `/v3/projects/6428f3104efa408ea802fcd5aab3162b/users/e8a4dc820cd9429f8c912c40b2b7e655/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.801257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f6f15-389f-4e3e-a0eb-72f76c7df6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.801856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f6f15-389f-4e3e-a0eb-72f76c7df6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.802139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f6f15-389f-4e3e-a0eb-72f76c7df6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.802466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f6f15-389f-4e3e-a0eb-72f76c7df6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.803152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f6f15-389f-4e3e-a0eb-72f76c7df6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.804811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f6f15-389f-4e3e-a0eb-72f76c7df6fc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.816140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6428f3104efa408ea802fcd5aab3162b, user_id=e8a4dc820cd9429f8c912c40b2b7e655, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.817605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 806/1604] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:48.819971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.822374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912bde-e2c6-49e1-88aa-5321b360771f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.822704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912bde-e2c6-49e1-88aa-5321b360771f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.823006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912bde-e2c6-49e1-88aa-5321b360771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.842280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-65912bde-e2c6-49e1-88aa-5321b360771f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:48.843669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-65912bde-e2c6-49e1-88aa-5321b360771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:48.851491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 807/1605] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:48.852267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912bde-e2c6-49e1-88aa-5321b360771f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.862022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912bde-e2c6-49e1-88aa-5321b360771f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:48.867515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:48.881115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:48.881115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:48.881115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] PATH_INFO: `/v3/projects/99f0beb902e4472eae20f0e6c8dc3340/users/b9252893998a490689497697663ca4d2/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:48.883524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f0beb902e4472eae20f0e6c8dc3340, user_id=b9252893998a490689497697663ca4d2, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:48.886164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:48.940759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 799/1606] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:48 2022] PUT /identity/v3/projects/6428f3104efa408ea802fcd5aab3162b/users/e8a4dc820cd9429f8c912c40b2b7e655/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:48.941565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:48.946106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:48.998018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 808/1607] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:48 2022] PUT /identity/v3/projects/99f0beb902e4472eae20f0e6c8dc3340/users/b9252893998a490689497697663ca4d2/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:48.998873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.004258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d8733-3e4c-44c2-a3d6-6d646763e8ab admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.031920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:49.058346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf141eb-4ffc-4d26-b08e-cb111a71ee48 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.060241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc67e1-1a97-485c-b4bb-c47c9d77f56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.061070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc67e1-1a97-485c-b4bb-c47c9d77f56f admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.061924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc67e1-1a97-485c-b4bb-c47c9d77f56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.062195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc67e1-1a97-485c-b4bb-c47c9d77f56f admin admin] PATH_INFO: `/v3/projects/43ef4ba417014205b91496a8037540ed/users/281e6cd581c54901b8feaa291db588eb/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.063233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.063858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.064254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.064440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.065088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.066836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.076836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc67e1-1a97-485c-b4bb-c47c9d77f56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ef4ba417014205b91496a8037540ed, user_id=281e6cd581c54901b8feaa291db588eb, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.077258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 809/1608] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:49.079647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc67e1-1a97-485c-b4bb-c47c9d77f56f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.081415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.090696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcbf1f-d862-4cee-a1ed-3306f3864529 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.095210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.095841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.096379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.100460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] PATH_INFO: `/v3/projects/e7a3075a9e3a47a4bcb9136ce8303173/users/751f05f1ff6c4593870073d8f47e0db9/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.111259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a3075a9e3a47a4bcb9136ce8303173, user_id=751f05f1ff6c4593870073d8f47e0db9, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.113907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.153838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 800/1609] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:48 2022] PUT /identity/v3/projects/43ef4ba417014205b91496a8037540ed/users/281e6cd581c54901b8feaa291db588eb/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:49.155324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152de0c9-c58e-47af-9b5e-9571cb8e8a5b None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.155609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152de0c9-c58e-47af-9b5e-9571cb8e8a5b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.155878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152de0c9-c58e-47af-9b5e-9571cb8e8a5b None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.157188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 801/1610] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 12:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:29:49.157847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152de0c9-c58e-47af-9b5e-9571cb8e8a5b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.184853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 810/1611] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:49 2022] PUT /identity/v3/projects/e7a3075a9e3a47a4bcb9136ce8303173/users/751f05f1ff6c4593870073d8f47e0db9/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:49.185756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.221295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152de0c9-c58e-47af-9b5e-9571cb8e8a5b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.243788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975bd02-f5b3-4e1b-81d6-4ed87524d0c3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.248056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29724475-6f41-497b-866c-e38ff56ac423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.248665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724475-6f41-497b-866c-e38ff56ac423 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.248945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724475-6f41-497b-866c-e38ff56ac423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.249275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724475-6f41-497b-866c-e38ff56ac423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.262986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:49.282120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded86d29-fe3e-486d-ac39-0823b5093b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.282875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded86d29-fe3e-486d-ac39-0823b5093b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.283154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded86d29-fe3e-486d-ac39-0823b5093b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.283425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded86d29-fe3e-486d-ac39-0823b5093b56 admin admin] PATH_INFO: `/v3/projects/b0950bef7bcc479998967ea1df55fe9d/users/341990278d184a298c411bbae2c8474d/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.297694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded86d29-fe3e-486d-ac39-0823b5093b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0950bef7bcc479998967ea1df55fe9d, user_id=341990278d184a298c411bbae2c8474d, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.300386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded86d29-fe3e-486d-ac39-0823b5093b56 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.312033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29724475-6f41-497b-866c-e38ff56ac423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.313537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29724475-6f41-497b-866c-e38ff56ac423 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.323163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 811/1612] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:49.324491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904edb-9e77-4bf2-aec9-50e617e8e333 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.331697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904edb-9e77-4bf2-aec9-50e617e8e333 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.331697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904edb-9e77-4bf2-aec9-50e617e8e333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.353469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-91904edb-9e77-4bf2-aec9-50e617e8e333 None None] MFA Rules not processed for user `e8a4dc820cd9429f8c912c40b2b7e655`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:49.354686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-91904edb-9e77-4bf2-aec9-50e617e8e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.392136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 802/1613] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:49 2022] PUT /identity/v3/projects/b0950bef7bcc479998967ea1df55fe9d/users/341990278d184a298c411bbae2c8474d/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:49.393541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6db98-3b5e-427f-82ad-1988c8ef62b1 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.393825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6db98-3b5e-427f-82ad-1988c8ef62b1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.394099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6db98-3b5e-427f-82ad-1988c8ef62b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.407355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:49.413420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-91b6db98-3b5e-427f-82ad-1988c8ef62b1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:49.414782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-91b6db98-3b5e-427f-82ad-1988c8ef62b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.434682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 812/1614] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:49.438955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904edb-9e77-4bf2-aec9-50e617e8e333 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.449376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904edb-9e77-4bf2-aec9-50e617e8e333 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.502943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 803/1615] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:49.504417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ee03-a21c-4675-a65d-0bff31d6bce8 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.504782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ee03-a21c-4675-a65d-0bff31d6bce8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.505128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ee03-a21c-4675-a65d-0bff31d6bce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.516435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bebb3b-910d-44b4-bb31-899a253ae1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.517093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bebb3b-910d-44b4-bb31-899a253ae1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.517370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bebb3b-910d-44b4-bb31-899a253ae1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.517638 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bebb3b-910d-44b4-bb31-899a253ae1c3 admin admin] PATH_INFO: `/v3/projects/03762335be8c4eb5b1eeb58da678b2b9/users/b3b3ab610ea142689067612f78a221b3/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.519628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bebb3b-910d-44b4-bb31-899a253ae1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03762335be8c4eb5b1eeb58da678b2b9, user_id=b3b3ab610ea142689067612f78a221b3, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.522510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bebb3b-910d-44b4-bb31-899a253ae1c3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.525183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-34c9ee03-a21c-4675-a65d-0bff31d6bce8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:49.526417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-34c9ee03-a21c-4675-a65d-0bff31d6bce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.533137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 813/1616] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:49 2022] PUT /identity/v3/projects/03762335be8c4eb5b1eeb58da678b2b9/users/b3b3ab610ea142689067612f78a221b3/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:49.538791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60693-edcd-4aaf-944a-b1731e629b77 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.538791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60693-edcd-4aaf-944a-b1731e629b77 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.538791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc60693-edcd-4aaf-944a-b1731e629b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.574533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9fc60693-edcd-4aaf-944a-b1731e629b77 None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:49.575922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9fc60693-edcd-4aaf-944a-b1731e629b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.587361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 804/1617] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:49.588695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ea188-b518-4fe0-8b1c-40db33136529 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.588963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ea188-b518-4fe0-8b1c-40db33136529 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.589296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ea188-b518-4fe0-8b1c-40db33136529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.608751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-dd0ea188-b518-4fe0-8b1c-40db33136529 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:49.609956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-dd0ea188-b518-4fe0-8b1c-40db33136529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.622319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 805/1618] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:49.623678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.623957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.624231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.657687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] MFA Rules not processed for user `341990278d184a298c411bbae2c8474d`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:49.658941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.678441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 814/1619] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:49.679325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60693-edcd-4aaf-944a-b1731e629b77 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.695038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc60693-edcd-4aaf-944a-b1731e629b77 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.700274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba0503-3133-4ca5-a38c-52612810a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.700879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba0503-3133-4ca5-a38c-52612810a5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.701208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba0503-3133-4ca5-a38c-52612810a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.701618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba0503-3133-4ca5-a38c-52612810a5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.702276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba0503-3133-4ca5-a38c-52612810a5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.704234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba0503-3133-4ca5-a38c-52612810a5ec admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.710093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:49.732111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 806/1620] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:49.733092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.734355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.789436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2b33-b895-4b5c-82b6-909c44134e92 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.793100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.793703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.793981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.794252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.803327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.804826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.816504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 807/1621] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:49.817148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 815/1622] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:49.818562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.819000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e52b86-4d5d-4336-bfd8-7a0212b8fbfd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.819566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e52b86-4d5d-4336-bfd8-7a0212b8fbfd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.819953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e52b86-4d5d-4336-bfd8-7a0212b8fbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.828152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33095ca-93f6-4055-b6be-d8d997a95932 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.855582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d9e52b86-4d5d-4336-bfd8-7a0212b8fbfd None None] MFA Rules not processed for user `b3b3ab610ea142689067612f78a221b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:49.857148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d9e52b86-4d5d-4336-bfd8-7a0212b8fbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:49.886112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.886751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.887029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.887296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.890959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:49.948122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 816/1623] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:49.949053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e52b86-4d5d-4336-bfd8-7a0212b8fbfd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:49.962752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e52b86-4d5d-4336-bfd8-7a0212b8fbfd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:49.967355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:49.967736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:49.968014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:49.968450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:49.978620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:49.980095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:49.990877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 808/1624] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:49.992164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.039269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:50.044981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.050841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.057633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 817/1625] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:50.058373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.067755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362e7f9-602f-4e69-8901-846ebf2e75ba service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.073822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.074430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.074731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.075003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.126611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca650992-ffc8-4acf-916f-fd540ec12fc3 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.130722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.131881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.131881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.132157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.144820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.146913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.157603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 809/1626] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:50.158503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.170758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b445a1f-d356-433a-ae6f-9c01a7a18bfd service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.176026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.176632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.176963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.178240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.179544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.181802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.199785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.201616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.213292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 818/1627] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:50.214243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.225497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff75e8-e7a7-4796-82d4-286741381325 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.230798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.232294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.232774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.233146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.244255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:50.248136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.250070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.262564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 819/1628] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:50.277989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 810/1629] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:50 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:50.282036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.299832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0302147-6044-445e-aee0-3d246383f4ff service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.306781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.307613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.307869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.308170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.308800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.310395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.322808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 820/1630] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:50.326613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.340881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517836a4-8a89-4694-920e-c84828b3f879 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.348048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.349591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.349591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.350066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] PATH_INFO: `/v3/projects/7ce4da6472f3402a991a68308a35e88b/users/3794e2de476948b9bec53b7e7d8dead4/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.395586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce4da6472f3402a991a68308a35e88b, user_id=3794e2de476948b9bec53b7e7d8dead4, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.398549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.502761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 811/1631] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:50 2022] PUT /identity/v3/projects/7ce4da6472f3402a991a68308a35e88b/users/3794e2de476948b9bec53b7e7d8dead4/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:29:50.507604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.523561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc76514-489b-4594-83d8-d13ad76438ec admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.558134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.652537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa165e-8490-4d48-854f-a26ed6a88820 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.665819 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.665819 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.666009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.666258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.667437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.668541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.689600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0d11-c59e-4c14-9659-369e90349d20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.690236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0d11-c59e-4c14-9659-369e90349d20 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.691052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 821/1632] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:50.691907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:50.692288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0d11-c59e-4c14-9659-369e90349d20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.692595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0d11-c59e-4c14-9659-369e90349d20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.702783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8b6e1-5d83-4356-a6ee-c1325cc3e9b0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:50.707844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a6da4-4a03-45cf-878f-ba8327fb1f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:50.708711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a6da4-4a03-45cf-878f-ba8327fb1f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.709094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a6da4-4a03-45cf-878f-ba8327fb1f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.709568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a6da4-4a03-45cf-878f-ba8327fb1f14 admin admin] PATH_INFO: `/v3/projects/7ce4da6472f3402a991a68308a35e88b/users/3794e2de476948b9bec53b7e7d8dead4/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.711614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a6da4-4a03-45cf-878f-ba8327fb1f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce4da6472f3402a991a68308a35e88b, user_id=3794e2de476948b9bec53b7e7d8dead4, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.714462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a6da4-4a03-45cf-878f-ba8327fb1f14 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.735683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 822/1633] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:50 2022] PUT /identity/v3/projects/7ce4da6472f3402a991a68308a35e88b/users/3794e2de476948b9bec53b7e7d8dead4/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:50.741005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dba64-290b-48f8-b423-4936fa207073 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.741196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dba64-290b-48f8-b423-4936fa207073 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.741464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dba64-290b-48f8-b423-4936fa207073 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.747798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 823/1634] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 12:29:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:29:50.750879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194065-fa56-4559-bc60-f12978b6c525 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.754866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194065-fa56-4559-bc60-f12978b6c525 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.754866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75194065-fa56-4559-bc60-f12978b6c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.785833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:50.788794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-75194065-fa56-4559-bc60-f12978b6c525 None None] MFA Rules not processed for user `3794e2de476948b9bec53b7e7d8dead4`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:50.790316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-75194065-fa56-4559-bc60-f12978b6c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:50.799838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0d11-c59e-4c14-9659-369e90349d20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:50.803262 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0d11-c59e-4c14-9659-369e90349d20 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:50.816515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 812/1635] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:50.819475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796dd0-f616-4f72-9e21-6cf545027648 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.819919 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796dd0-f616-4f72-9e21-6cf545027648 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.820335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796dd0-f616-4f72-9e21-6cf545027648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.854033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0b796dd0-f616-4f72-9e21-6cf545027648 None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:50.855423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0b796dd0-f616-4f72-9e21-6cf545027648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:50.894688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 824/1636] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:50.955591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 813/1637] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:50.962268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b28d-c24b-4aa3-92cc-eb54e6a9b4fd None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.966869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b28d-c24b-4aa3-92cc-eb54e6a9b4fd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.966869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b28d-c24b-4aa3-92cc-eb54e6a9b4fd None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.966869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 825/1638] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 12:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:29:50.970850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f1cd-553f-4bea-bb5e-7846046f00ca None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:50.971213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f1cd-553f-4bea-bb5e-7846046f00ca None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:50.971497 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f1cd-553f-4bea-bb5e-7846046f00ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:50.992655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b0c3f1cd-553f-4bea-bb5e-7846046f00ca None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:51.002589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b0c3f1cd-553f-4bea-bb5e-7846046f00ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:51.003588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 814/1639] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:51.223011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b28d-c24b-4aa3-92cc-eb54e6a9b4fd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:51.314730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b28d-c24b-4aa3-92cc-eb54e6a9b4fd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:51.319057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:51.319732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:51.320006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:51.320277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:51.328278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:51.330937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:51.332415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:51.344522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 826/1640] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:51.388094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f1cd-553f-4bea-bb5e-7846046f00ca None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:51.390012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:51.407690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89066abd-e418-434e-b814-7fe3fa415614 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:51.422063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d0a0-14ee-491b-8aed-0b9aa31127c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:51.423073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d0a0-14ee-491b-8aed-0b9aa31127c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:51.424258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d0a0-14ee-491b-8aed-0b9aa31127c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:51.432596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d0a0-14ee-491b-8aed-0b9aa31127c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:51.448730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d0a0-14ee-491b-8aed-0b9aa31127c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:51.450356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d0a0-14ee-491b-8aed-0b9aa31127c0 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:51.493973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 827/1641] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:51.590175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f1cd-553f-4bea-bb5e-7846046f00ca None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:51.611603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:51.612331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:51.615592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:51.615592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] PATH_INFO: `/v3/users/e8a4dc820cd9429f8c912c40b2b7e655` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:51.616595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a4dc820cd9429f8c912c40b2b7e655)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:51.621726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:51.672583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4a53-0341-4332-a86a-12e876b43351 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:51.672871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4a53-0341-4332-a86a-12e876b43351 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:51.673160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4a53-0341-4332-a86a-12e876b43351 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:51.674845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 828/1642] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 12:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:29:51.681998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39689b-4dd2-4b3c-b691-6bd6addd25da None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:51.682285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39689b-4dd2-4b3c-b691-6bd6addd25da None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:51.682556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39689b-4dd2-4b3c-b691-6bd6addd25da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:51.716433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ac39689b-4dd2-4b3c-b691-6bd6addd25da None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:51.723539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ac39689b-4dd2-4b3c-b691-6bd6addd25da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:51.732394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 829/1643] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:29:51.748970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39689b-4dd2-4b3c-b691-6bd6addd25da None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:51.759941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39689b-4dd2-4b3c-b691-6bd6addd25da None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:51.763452 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:51.764091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:51.764372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:51.764639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:51.776115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:51.781336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:51.802738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 830/1644] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:29:51.855096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a4dc820cd9429f8c912c40b2b7e655'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:51.855737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a4dc820cd9429f8c912c40b2b7e655'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:51.856542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a4dc820cd9429f8c912c40b2b7e655'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:51.866063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a4dc820cd9429f8c912c40b2b7e655'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:51.879133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-695ca8f4-830e-423b-853a-eb8db871c131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a4dc820cd9429f8c912c40b2b7e655'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:51.883130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:51.911038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:51.917883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c3845-3cbd-4f03-baf2-8bf9699bcf5e service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:51.929941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:51.930809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:51.931046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:51.931326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] PATH_INFO: `/v3/users/7750a81cb89346ecb3634daa137d0da3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:51.932545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7750a81cb89346ecb3634daa137d0da3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:51.934936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: WARNING py.warnings [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:51.934936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: warnings.warn(msg) Aug 24 12:29:51.934936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: Aug 24 12:29:51.938765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:51.995904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 815/1645] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:29:51 2022] DELETE /identity/v3/users/e8a4dc820cd9429f8c912c40b2b7e655 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:52.000416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:52.000959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:52.001552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:52.030257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:52.031511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:52.099328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 816/1646] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:29:52.109798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:52.112050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:52.198611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25f55-ac35-46f7-86f7-124994abd229 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:52.206678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352ba93-f1ff-44f6-a457-abedc665ce46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:52.478189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352ba93-f1ff-44f6-a457-abedc665ce46 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:52.478189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352ba93-f1ff-44f6-a457-abedc665ce46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:52.478189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352ba93-f1ff-44f6-a457-abedc665ce46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:52.478189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352ba93-f1ff-44f6-a457-abedc665ce46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:52.478189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352ba93-f1ff-44f6-a457-abedc665ce46 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:52.478189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 817/1647] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:29:52.478917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7750a81cb89346ecb3634daa137d0da3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:52.478917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7750a81cb89346ecb3634daa137d0da3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:52.478917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7750a81cb89346ecb3634daa137d0da3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:52.478917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7750a81cb89346ecb3634daa137d0da3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:52.478917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7750a81cb89346ecb3634daa137d0da3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:52.482309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 831/1648] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:29:51 2022] DELETE /identity/v3/users/7750a81cb89346ecb3634daa137d0da3 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:29:52.482693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:52.482693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:52.482693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:52.482693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:52.482693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:52.562689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 818/1649] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:52.572958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:52.575383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:52.665132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cac4-c0d9-41f7-a87f-29c85e22e876 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:52.669621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:52.670388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:52.670681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:52.670999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:52.684261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:52.686099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:52.703321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 832/1650] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:53.268939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:53.270450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:53.319992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03531-92d9-49a1-8571-ab9e674c4699 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:53.350963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:53.351632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:53.351910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:53.352185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] PATH_INFO: `/v3/projects/6428f3104efa408ea802fcd5aab3162b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:53.360699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6428f3104efa408ea802fcd5aab3162b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:53.361758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: WARNING py.warnings [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:53.361758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: warnings.warn(msg) Aug 24 12:29:53.361758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: Aug 24 12:29:53.365902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:53.581135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6428f3104efa408ea802fcd5aab3162b', 'request_id': 'req-57f8abab-3851-4416-9288-0ea984ec9d01'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:53.640803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6428f3104efa408ea802fcd5aab3162b', 'request_id': 'req-57f8abab-3851-4416-9288-0ea984ec9d01'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:53.645664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:53.682608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6428f3104efa408ea802fcd5aab3162b', 'request_id': 'req-57f8abab-3851-4416-9288-0ea984ec9d01'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:53.779941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] The token cache is being invalidate because project 6428f3104efa408ea802fcd5aab3162b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:29:53.780238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:53.781041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:53.790377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 819/1651] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:29:53 2022] DELETE /identity/v3/projects/6428f3104efa408ea802fcd5aab3162b => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:53.794501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:53.813256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b31640-999c-4866-8944-91513ebea340 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:53.876119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:53.934577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:53.935275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:53.935554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:53.935824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] PATH_INFO: `/v3/users/3794e2de476948b9bec53b7e7d8dead4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:53.943606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3794e2de476948b9bec53b7e7d8dead4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:53.945578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:54.202811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3794e2de476948b9bec53b7e7d8dead4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.206793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3794e2de476948b9bec53b7e7d8dead4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.319937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3794e2de476948b9bec53b7e7d8dead4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.342868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3794e2de476948b9bec53b7e7d8dead4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.346185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-26a2b8e1-0cc5-4677-9135-8c683c35af3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3794e2de476948b9bec53b7e7d8dead4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.370937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 833/1652] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:29:53 2022] DELETE /identity/v3/users/3794e2de476948b9bec53b7e7d8dead4 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:54.424501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:54.426109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:54.463389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8abab-3851-4416-9288-0ea984ec9d01 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:54.567063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:54.568251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:54.568534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:54.568805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] PATH_INFO: `/v3/projects/94e584086ede4dcfb7fa3002b1091b8e` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:54.570429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e584086ede4dcfb7fa3002b1091b8e)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:54.572565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:54.737129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e584086ede4dcfb7fa3002b1091b8e', 'request_id': 'req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.746060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e584086ede4dcfb7fa3002b1091b8e', 'request_id': 'req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.763144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e584086ede4dcfb7fa3002b1091b8e', 'request_id': 'req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.869652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] The token cache is being invalidate because project 94e584086ede4dcfb7fa3002b1091b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:29:54.871105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.871105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2f9e7f6a-868b-41cf-9862-d7e4856f11a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:54.875772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 820/1653] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:29:54 2022] DELETE /identity/v3/projects/94e584086ede4dcfb7fa3002b1091b8e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:54.877586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99501-4da5-46c9-8f9b-3d8b0a5fd7df None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:54.878020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99501-4da5-46c9-8f9b-3d8b0a5fd7df None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:54.878020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99501-4da5-46c9-8f9b-3d8b0a5fd7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:54.914876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fcc99501-4da5-46c9-8f9b-3d8b0a5fd7df None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:54.918551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fcc99501-4da5-46c9-8f9b-3d8b0a5fd7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:55.106358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 834/1654] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:55.107172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57a190-939f-4ff7-98ce-756d4f71f513 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:55.109278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57a190-939f-4ff7-98ce-756d4f71f513 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:55.109278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57a190-939f-4ff7-98ce-756d4f71f513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:55.147252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2b57a190-939f-4ff7-98ce-756d4f71f513 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:55.152981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2b57a190-939f-4ff7-98ce-756d4f71f513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:55.174810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 821/1655] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:55.178098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc99501-4da5-46c9-8f9b-3d8b0a5fd7df None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:55.191959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc99501-4da5-46c9-8f9b-3d8b0a5fd7df None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:55.200160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:55.200798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:55.201230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:55.201520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:55.202184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:55.202485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:55.204282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:55.233990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 835/1656] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:55.238266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57a190-939f-4ff7-98ce-756d4f71f513 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:55.248959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57a190-939f-4ff7-98ce-756d4f71f513 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:55.265318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:55.266035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:55.266385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:55.266745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:55.268081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:55.273179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:55.472290 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 822/1657] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:55 2022] POST /identity/v3/projects => generated 373 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:55.478391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:55.493803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543596a0-cfe6-4460-bc4b-0bce47e1f605 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:55.568548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:55.570736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:55.650273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:55.651045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:55.651518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:55.654065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:55.654065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:55.654418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:55.659268 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:55.772255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 836/1658] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:55 2022] POST /identity/v3/users => generated 287 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:55.778392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:55.807944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:55.809665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f2dc-2e19-431a-a3bf-4c735d12d194 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:55.821341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:55.824203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:55.824706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:55.825125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:55.826972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283eef69-815f-4752-86ff-a4b472154a16 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:55.827592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:55.831950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:55.848080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:55.849885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:55.850644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:55.854628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] PATH_INFO: `/v3/projects/7ce4da6472f3402a991a68308a35e88b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:55.856921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce4da6472f3402a991a68308a35e88b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:55.860685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:55.863909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 837/1659] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:55.873143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:55.915810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cc5ae-9d61-4e51-bba0-3a76992137d7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:55.919251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:55.920033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:55.920451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:55.920944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:55.921987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:55.923755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:55.955534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 838/1660] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:55.956760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:55.964393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:55.993494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15d563-c9a4-4ed1-a12e-51ae2823b592 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:56.008600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:56.009266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:56.009551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:56.009823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] PATH_INFO: `/v3/users/6a9228ec00ef4ff5b2a90992499f4d8f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:56.011129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9228ec00ef4ff5b2a90992499f4d8f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:56.015439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:56.094035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce4da6472f3402a991a68308a35e88b', 'request_id': 'req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.102543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce4da6472f3402a991a68308a35e88b', 'request_id': 'req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.111010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce4da6472f3402a991a68308a35e88b', 'request_id': 'req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.244600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] The token cache is being invalidate because project 7ce4da6472f3402a991a68308a35e88b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:29:56.244904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.245714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.251761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 823/1661] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:29:55 2022] DELETE /identity/v3/projects/7ce4da6472f3402a991a68308a35e88b => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:29:56.251761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:56.256574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:56.332725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9228ec00ef4ff5b2a90992499f4d8f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.336044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9228ec00ef4ff5b2a90992499f4d8f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.415738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd521cd-4730-42ce-b00e-6f2fdcfb4fb8 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:56.429602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bef400-d49a-4639-8686-c1779579d45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:56.430037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef400-d49a-4639-8686-c1779579d45c admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:56.430316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef400-d49a-4639-8686-c1779579d45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:56.430595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef400-d49a-4639-8686-c1779579d45c admin admin] PATH_INFO: `/v3/projects/d0274cb0d0e14d25b283cb912f1c4927/users/29fef7a337dc470e9e7988927a91d921/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:56.468072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bef400-d49a-4639-8686-c1779579d45c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0274cb0d0e14d25b283cb912f1c4927, user_id=29fef7a337dc470e9e7988927a91d921, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:56.468879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9228ec00ef4ff5b2a90992499f4d8f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.472349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bef400-d49a-4639-8686-c1779579d45c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:56.593468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9228ec00ef4ff5b2a90992499f4d8f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.595937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d8a43317-4b0c-4b0b-9303-8c1b6360cee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9228ec00ef4ff5b2a90992499f4d8f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:56.624156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 839/1662] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:29:55 2022] DELETE /identity/v3/users/6a9228ec00ef4ff5b2a90992499f4d8f => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:56.626266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd320-03e8-4509-85e3-b032374942b0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:56.627143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd320-03e8-4509-85e3-b032374942b0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:56.629568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fd320-03e8-4509-85e3-b032374942b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:56.672591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bc0fd320-03e8-4509-85e3-b032374942b0 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:56.674104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bc0fd320-03e8-4509-85e3-b032374942b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:56.720547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 824/1663] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:56 2022] PUT /identity/v3/projects/d0274cb0d0e14d25b283cb912f1c4927/users/29fef7a337dc470e9e7988927a91d921/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:56.723750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37fefa-9e59-4735-b436-05f84cfc7fe4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:56.729446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37fefa-9e59-4735-b436-05f84cfc7fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:56.734838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37fefa-9e59-4735-b436-05f84cfc7fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:56.764975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 840/1664] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:56.766499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:56.767119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:56.767618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:56.793559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ec37fefa-9e59-4735-b436-05f84cfc7fe4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:56.795077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ec37fefa-9e59-4735-b436-05f84cfc7fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:56.815246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] MFA Rules not processed for user `29fef7a337dc470e9e7988927a91d921`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:56.816858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:56.930082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 825/1665] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:56.940888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:56.940888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:56.941112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:56.965711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 841/1666] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:56.968196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:56.972089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:56.996582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad449af2-8241-4626-ad76-d32455132cf8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.003110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.003953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.004383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.004805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.005740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.009242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:57.011805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:57.015296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:57.040499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 826/1667] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:57.052744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:57.055276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:57.112859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 842/1668] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:56 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:29:57.114175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:57.186220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0904b-dfb3-47ab-a9b3-360e96c19d48 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.297079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbda78e-1d16-4953-b413-479bd5ceb225 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.320227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:57.341098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.347139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.348697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.349545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.357196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:57.359103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.360751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:57.378544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.378891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:57.379197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.385892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.386696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.411385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 843/1669] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:57.412768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:57.417480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.425252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:57.451068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1bcf1-5d26-4dcb-9f71-2b291d643199 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.455868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 827/1670] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:57.456942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:57.468056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.470470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.473849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.479708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.479708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.483172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:57.493227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1cdc4-827e-4c74-9f9f-a6d5b9166466 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.528001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.530737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.531204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.533065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.542867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.542867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:57.667530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 844/1671] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:57 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:57.671561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:57.686935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213df04-cedd-4a66-a9fd-62eae47f8973 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.700862 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 828/1672] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:57 2022] POST /identity/v3/projects => generated 393 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:57.705954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:57.706900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.707534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.707813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.708085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.709402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.711806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:57.749341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 845/1673] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:57.752384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:57.784745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d8305-bc3f-4701-becc-52031674090a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.885988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875c94-0d64-4941-aaf8-d9086eb3c318 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:57.901785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.902429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.902819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.903001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.903719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.906231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:57.959360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:57.960066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:57.960390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:57.960716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] PATH_INFO: `/v3/projects/b75be27c6e1445b1b49ba998a9c4a216/users/4b72b4e526a44c4b8b2730be2058e39d/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:57.980510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75be27c6e1445b1b49ba998a9c4a216, user_id=4b72b4e526a44c4b8b2730be2058e39d, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:57.985147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:58.032564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 829/1674] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:57 2022] POST /identity/v3/users => generated 297 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:58.033719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:58.054732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38920287-e555-4bfc-8afd-3a225f0330f1 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:58.071012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:58.071942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:58.072931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:58.074297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:58.075684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:58.078237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:58.109805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 846/1675] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:57 2022] PUT /identity/v3/projects/b75be27c6e1445b1b49ba998a9c4a216/users/4b72b4e526a44c4b8b2730be2058e39d/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:29:58.115646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:58.153341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 830/1676] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:58.154785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:58.182241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315b75e-9dc1-46bc-9107-0aef9d9351e0 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:58.330524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adb628-d993-435b-bfe5-418582427a5b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:58.348191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:58.359553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:58.359553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:58.359553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:58.359553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:58.359553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:58.365707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:58.366340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:58.366617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:58.366945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:58.367713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:58.377388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:58.405721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 831/1677] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:58.408792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:58.445812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dca63-604d-414e-8101-1775692b8e26 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:58.445812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e387a75-9ca7-41ec-9c91-4146e295a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:58.446038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 847/1678] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:29:58.447140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e387a75-9ca7-41ec-9c91-4146e295a01f admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:58.447665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e387a75-9ca7-41ec-9c91-4146e295a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:58.448397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e387a75-9ca7-41ec-9c91-4146e295a01f admin admin] PATH_INFO: `/v3/projects/70ef0e2e49654a40ad801587b8427b00/users/5242265288ff407892038bbf7f294d71/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:58.451617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:58.502743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e387a75-9ca7-41ec-9c91-4146e295a01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ef0e2e49654a40ad801587b8427b00, user_id=5242265288ff407892038bbf7f294d71, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:58.503271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3afe5f-9edb-4eb6-bd84-f18b689957c5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:58.507764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e387a75-9ca7-41ec-9c91-4146e295a01f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:58.528171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32259934-bc0f-46bc-8153-941f43e0b0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:58.528868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259934-bc0f-46bc-8153-941f43e0b0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:58.529157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259934-bc0f-46bc-8153-941f43e0b0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:58.529439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259934-bc0f-46bc-8153-941f43e0b0b0 admin admin] PATH_INFO: `/v3/projects/b75be27c6e1445b1b49ba998a9c4a216/users/4b72b4e526a44c4b8b2730be2058e39d/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:58.540235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32259934-bc0f-46bc-8153-941f43e0b0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75be27c6e1445b1b49ba998a9c4a216, user_id=4b72b4e526a44c4b8b2730be2058e39d, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:58.549990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32259934-bc0f-46bc-8153-941f43e0b0b0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:58.594798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 848/1679] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:58 2022] PUT /identity/v3/projects/b75be27c6e1445b1b49ba998a9c4a216/users/4b72b4e526a44c4b8b2730be2058e39d/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:58.601853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 832/1680] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:58 2022] PUT /identity/v3/projects/70ef0e2e49654a40ad801587b8427b00/users/5242265288ff407892038bbf7f294d71/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:58.610228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:58.610228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:58.610228 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:58.625059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f144b7a-c3d8-480d-ad6a-2f10fa9c5b37 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:58.625706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f144b7a-c3d8-480d-ad6a-2f10fa9c5b37 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:58.626167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f144b7a-c3d8-480d-ad6a-2f10fa9c5b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:58.696749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5f144b7a-c3d8-480d-ad6a-2f10fa9c5b37 None None] MFA Rules not processed for user `5242265288ff407892038bbf7f294d71`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:58.701017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5f144b7a-c3d8-480d-ad6a-2f10fa9c5b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:58.711137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] MFA Rules not processed for user `4b72b4e526a44c4b8b2730be2058e39d`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:58.712593 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:58.848066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 833/1681] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:58.852977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f144b7a-c3d8-480d-ad6a-2f10fa9c5b37 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:58.920064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 849/1682] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:29:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:58.939300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:58.939598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:58.939692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:58.992886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f144b7a-c3d8-480d-ad6a-2f10fa9c5b37 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:59.016577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:59.016577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:59.016577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:59.016577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:59.016927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:59.016927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:59.068448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:59.088799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87df8f1-c286-491b-87dc-7ebd5d269a6e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:59.099462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:59.100198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:59.100572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:59.100909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:59.122838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:59.128180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:59.139481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 834/1683] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:58 2022] POST /identity/v3/projects => generated 393 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:59.140943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:59.144675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 850/1684] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:59.275754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe048-37bc-4c08-a04d-01393c76edaa admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:59.287158 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:59.294350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:59.294977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:59.295823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:59.296233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:59.301095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:59.416004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 835/1685] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:59 2022] POST /identity/v3/users => generated 303 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:59.419448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:59.433836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd800753-e077-48f9-a011-5dc5d135491e service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:59.453578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:59.454582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:59.455130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:59.455598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:59.456437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:59.463502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:59.497558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 851/1686] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:59.518783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:59.520306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:59.527415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:59.541617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65015a5-4ca5-4f98-b853-fc76f1456d3a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:59.558684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062859c-3c84-439f-8ef3-e1b655eaef1e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:59.566432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:59.569015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:59.578250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:59.578870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:59.579815 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:59.580238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] PATH_INFO: `/v3/projects/d6bc9523696d4a50a8ae80631a03a6ed` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:59.582100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6bc9523696d4a50a8ae80631a03a6ed)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:59.584108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:59.616150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:59.620517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] PATH_INFO: `/v3/projects/2a8300a4efa049638d148d246a7086f7/users/54bef09a96724d14a8e091110d958bec/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:59.669699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8300a4efa049638d148d246a7086f7, user_id=54bef09a96724d14a8e091110d958bec, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:59.682412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:59.803710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 836/1687] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:59 2022] PUT /identity/v3/projects/2a8300a4efa049638d148d246a7086f7/users/54bef09a96724d14a8e091110d958bec/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:59.808211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:59.865944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6bc9523696d4a50a8ae80631a03a6ed', 'request_id': 'req-62b33e92-5b18-4c12-bf62-264fe5203f84'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:59.879830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6bc9523696d4a50a8ae80631a03a6ed', 'request_id': 'req-62b33e92-5b18-4c12-bf62-264fe5203f84'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:59.903185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6bc9523696d4a50a8ae80631a03a6ed', 'request_id': 'req-62b33e92-5b18-4c12-bf62-264fe5203f84'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:59.980528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95ef4-8a65-4855-8733-c4bc1bbaadc6 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:00.037269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] The token cache is being invalidate because project d6bc9523696d4a50a8ae80631a03a6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:00.037842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:00.038933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:00.062929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 852/1688] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:29:59 2022] DELETE /identity/v3/projects/d6bc9523696d4a50a8ae80631a03a6ed => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:00.064987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:00.068030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:00.192858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:00.193567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:00.193849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:00.194123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:00.194833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:00.209597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:00.259491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 837/1689] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:00.260570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:00.270292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:00.292981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af303504-e2bb-4d12-8cc3-a52884c04567 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:00.293402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b33e92-5b18-4c12-bf62-264fe5203f84 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:00.299495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:00.300370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:00.300882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:00.301299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] PATH_INFO: `/v3/users/b3b3ab610ea142689067612f78a221b3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:00.304382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3b3ab610ea142689067612f78a221b3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:00.306459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:00.309631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae30639-ab72-4ced-9e97-2d813f653124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:00.309631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30639-ab72-4ced-9e97-2d813f653124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:00.310308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30639-ab72-4ced-9e97-2d813f653124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:00.310308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae30639-ab72-4ced-9e97-2d813f653124 admin admin] PATH_INFO: `/v3/projects/2a8300a4efa049638d148d246a7086f7/users/54bef09a96724d14a8e091110d958bec/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:00.312365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30639-ab72-4ced-9e97-2d813f653124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8300a4efa049638d148d246a7086f7, user_id=54bef09a96724d14a8e091110d958bec, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:00.315256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae30639-ab72-4ced-9e97-2d813f653124 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:00.337616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 838/1690] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:00 2022] PUT /identity/v3/projects/2a8300a4efa049638d148d246a7086f7/users/54bef09a96724d14a8e091110d958bec/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:00.351977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:00.351977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:00.351977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:00.400487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:00.433708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] MFA Rules not processed for user `54bef09a96724d14a8e091110d958bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:00.435362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:00.589895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3b3ab610ea142689067612f78a221b3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:00.589895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3b3ab610ea142689067612f78a221b3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:00.630064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 839/1691] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:00.651742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:00.653236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:00.685700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:00.728087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3b3ab610ea142689067612f78a221b3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:00.772186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3b3ab610ea142689067612f78a221b3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:00.773575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3b3ab610ea142689067612f78a221b3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:00.781922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90056b-91cf-4c20-96ed-34cf69cbdabf None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:00.790146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:00.791057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:00.791507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:00.792097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:00.794092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:00.809952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:00.811681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:00.812692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 853/1692] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:00 2022] DELETE /identity/v3/users/b3b3ab610ea142689067612f78a221b3 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:00.818975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:00.824985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:00.839812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 840/1693] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:30:00.997232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:00.999131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:01.023459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab62a52-178d-413c-b118-7b2223aca302 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:01.045148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb84c89-d9e7-4b5f-aa2a-cb8be68e6536 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:01.051035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:01.053803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:01.054814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:01.055599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] PATH_INFO: `/v3/users/29fef7a337dc470e9e7988927a91d921` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:01.057035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29fef7a337dc470e9e7988927a91d921)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:01.059879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:01.120016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0da0d8-7490-4078-9d1c-8ddd67b8caed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:01.120648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0da0d8-7490-4078-9d1c-8ddd67b8caed service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:01.121811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0da0d8-7490-4078-9d1c-8ddd67b8caed service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:01.121811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0da0d8-7490-4078-9d1c-8ddd67b8caed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:01.122975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2a0da0d8-7490-4078-9d1c-8ddd67b8caed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:01.255755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0da0d8-7490-4078-9d1c-8ddd67b8caed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:01.257229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0da0d8-7490-4078-9d1c-8ddd67b8caed service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:01.277330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 841/1694] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:30:01.314522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29fef7a337dc470e9e7988927a91d921'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:01.316621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29fef7a337dc470e9e7988927a91d921'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:01.451314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29fef7a337dc470e9e7988927a91d921'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:01.476914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:01.477212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:01.477564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:01.509278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29fef7a337dc470e9e7988927a91d921'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:01.511867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-77698b06-005c-4efa-a8bf-d23c298b5fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29fef7a337dc470e9e7988927a91d921'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:01.544011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] MFA Rules not processed for user `6b94f244190e4af089d070ad83806c38`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:01.548284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 854/1695] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:00 2022] DELETE /identity/v3/users/29fef7a337dc470e9e7988927a91d921 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:01.551544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:01.565986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:01.566279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:01.566761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:01.651347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:01.654561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:01.699559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:01.701403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 842/1696] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:01.718199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:01.725122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:01.897301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 855/1697] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:01.972251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33ce21-1949-4f0a-8505-af4422b02bff None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:01.988346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:01.989133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:01.989450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:01.989987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:02.009757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:02.015414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:02.032826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:02.040029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 843/1698] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:02.072719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:02.072719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:02.199548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3076f-66fa-4f54-9425-797e0e2b08e8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:02.203619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726548fe-4f8a-47c4-9f81-4fb147a9d4ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:02.204594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726548fe-4f8a-47c4-9f81-4fb147a9d4ec service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:02.205080 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726548fe-4f8a-47c4-9f81-4fb147a9d4ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:02.206104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726548fe-4f8a-47c4-9f81-4fb147a9d4ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:02.217830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726548fe-4f8a-47c4-9f81-4fb147a9d4ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:02.219816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726548fe-4f8a-47c4-9f81-4fb147a9d4ec service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:02.231595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 856/1699] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:02.914200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:03.290404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:03.291870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:03.314680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfc3d1-87b6-4aec-a521-117cce371be0 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:03.330030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:03.330934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:03.331363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:03.331777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] PATH_INFO: `/v3/projects/03762335be8c4eb5b1eeb58da678b2b9` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:03.333852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03762335be8c4eb5b1eeb58da678b2b9)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:03.339615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:03.427466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3db2c4-d676-48e3-8291-755afb084eaa None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:03.427466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3db2c4-d676-48e3-8291-755afb084eaa None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:03.427466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3db2c4-d676-48e3-8291-755afb084eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:03.492107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6c3db2c4-d676-48e3-8291-755afb084eaa None None] MFA Rules not processed for user `751f05f1ff6c4593870073d8f47e0db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:03.498681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6c3db2c4-d676-48e3-8291-755afb084eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:03.545347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03762335be8c4eb5b1eeb58da678b2b9', 'request_id': 'req-aae85dde-5b80-4b9b-9634-66a9177eb340'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:03.565092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03762335be8c4eb5b1eeb58da678b2b9', 'request_id': 'req-aae85dde-5b80-4b9b-9634-66a9177eb340'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:03.578607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03762335be8c4eb5b1eeb58da678b2b9', 'request_id': 'req-aae85dde-5b80-4b9b-9634-66a9177eb340'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:03.680499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] The token cache is being invalidate because project 03762335be8c4eb5b1eeb58da678b2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:03.680499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:03.682978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aae85dde-5b80-4b9b-9634-66a9177eb340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:03.692566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 844/1700] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:03 2022] DELETE /identity/v3/projects/03762335be8c4eb5b1eeb58da678b2b9 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:03.694185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:03.694473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:03.694774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:03.702123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:03.703065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 857/1701] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:03.719391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3db2c4-d676-48e3-8291-755afb084eaa None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:03.763634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3db2c4-d676-48e3-8291-755afb084eaa None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:03.765298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:03.766528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:03.931082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 845/1702] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:03.931980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:03.939620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:03.966105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96be0e-a71d-4b89-ad13-752f03feaeed None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:03.971470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:03.972953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:03.972953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:03.973131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] PATH_INFO: `/v3/projects/d0274cb0d0e14d25b283cb912f1c4927` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:03.975678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0274cb0d0e14d25b283cb912f1c4927)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:03.978444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:04.150833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:04.151483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:04.155005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:04.161418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:04.183926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:04.187874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0274cb0d0e14d25b283cb912f1c4927', 'request_id': 'req-891a62bf-377d-4f11-923e-070e419c1e1e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:04.204636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0274cb0d0e14d25b283cb912f1c4927', 'request_id': 'req-891a62bf-377d-4f11-923e-070e419c1e1e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:04.224832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0274cb0d0e14d25b283cb912f1c4927', 'request_id': 'req-891a62bf-377d-4f11-923e-070e419c1e1e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:04.325851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] The token cache is being invalidate because project d0274cb0d0e14d25b283cb912f1c4927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:04.325851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:04.325851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-891a62bf-377d-4f11-923e-070e419c1e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:04.325851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 846/1703] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:03 2022] DELETE /identity/v3/projects/d0274cb0d0e14d25b283cb912f1c4927 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:04.327492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0081b-f56c-40ba-861b-ec469f23ea0f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:04.327877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0081b-f56c-40ba-861b-ec469f23ea0f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:04.328244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0081b-f56c-40ba-861b-ec469f23ea0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:04.369096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:04.371654 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:04.398161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-75d0081b-f56c-40ba-861b-ec469f23ea0f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:04.402183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-75d0081b-f56c-40ba-861b-ec469f23ea0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:04.405964 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 858/1704] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:30:04.408915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:04.434603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b63f95-9924-4dc7-bca1-f739258fd524 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:04.471264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:04.729604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 847/1705] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:04.738053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0081b-f56c-40ba-861b-ec469f23ea0f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:04.765726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0081b-f56c-40ba-861b-ec469f23ea0f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:04.772358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:04.773801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:04.774129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:04.774406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] PATH_INFO: `/v3/users/4b72b4e526a44c4b8b2730be2058e39d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:04.775778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:04.776466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b72b4e526a44c4b8b2730be2058e39d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:04.777103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:04.780149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:04.780660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:04.781049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:04.781938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:04.782340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:04.789553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:04.834847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 848/1706] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:04.837828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:04.863291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59767a-88d0-42d0-851a-b42008cefeb2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:04.875573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:04.878280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:04.878280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:04.878280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:04.878280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:04.881403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:04.932335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:05.039876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 849/1707] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:04 2022] POST /identity/v3/projects => generated 369 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:05.049999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b72b4e526a44c4b8b2730be2058e39d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:05.050896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b72b4e526a44c4b8b2730be2058e39d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:05.051108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:05.191983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b72b4e526a44c4b8b2730be2058e39d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:05.232592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b72b4e526a44c4b8b2730be2058e39d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:05.240016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b72b4e526a44c4b8b2730be2058e39d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:05.265345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 859/1708] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:04 2022] DELETE /identity/v3/users/4b72b4e526a44c4b8b2730be2058e39d => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:05.281458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bc94d-2459-4543-8829-9034f0d0b1de admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:05.430072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:05.431942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:05.432454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:05.432751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:05.433410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:05.439995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:05.582544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 850/1709] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:05 2022] POST /identity/v3/users => generated 285 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:05.582638 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:05.583519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:05.611532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daf740-ab2d-429a-b041-f30865867670 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:05.630423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:05.631334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:05.631773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:05.632178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:05.632969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:05.634836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:05.662869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:05.664366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 860/1710] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:05.677721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ec06-7747-49c3-8c7a-9d9bf0a37103 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:05.705194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161e725-eba3-4875-8cb9-83b1cb94c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:05.707665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161e725-eba3-4875-8cb9-83b1cb94c0da admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:05.708992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161e725-eba3-4875-8cb9-83b1cb94c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:05.709399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161e725-eba3-4875-8cb9-83b1cb94c0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:05.712078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161e725-eba3-4875-8cb9-83b1cb94c0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:05.715647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161e725-eba3-4875-8cb9-83b1cb94c0da admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:05.751466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 851/1711] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:05.753240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:05.766397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932949-fccc-4b53-bd03-bf3f566c140f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:05.779050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:05.779858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:05.780134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:05.782612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] PATH_INFO: `/v3/projects/5a1327efac234cf9ba49a4fd1285305c/users/64a76dd851f44de48681960461c70d8e/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:05.823419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:05.846816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1327efac234cf9ba49a4fd1285305c, user_id=64a76dd851f44de48681960461c70d8e, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:05.855125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:05.973012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 861/1712] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:05 2022] PUT /identity/v3/projects/5a1327efac234cf9ba49a4fd1285305c/users/64a76dd851f44de48681960461c70d8e/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:05.974345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594270ec-267a-4624-ab7e-49be93f091a2 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:05.974832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594270ec-267a-4624-ab7e-49be93f091a2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:05.974934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594270ec-267a-4624-ab7e-49be93f091a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:06.034837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-594270ec-267a-4624-ab7e-49be93f091a2 None None] MFA Rules not processed for user `64a76dd851f44de48681960461c70d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:06.036648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-594270ec-267a-4624-ab7e-49be93f091a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:06.193998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 852/1713] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:06.199250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:06.324980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebbfd0-df20-4a2d-878b-b431bcf9c9bc admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:06.329763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:06.330389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:06.330727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:06.331012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:06.331641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:06.335160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:06.490942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 862/1714] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:06 2022] POST /identity/v3/projects => generated 367 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:06.493224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594270ec-267a-4624-ab7e-49be93f091a2 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:06.508415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594270ec-267a-4624-ab7e-49be93f091a2 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:06.631668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:06.632349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:06.632664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:06.633192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:06.633846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:06.636147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:06.763779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 853/1715] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:06 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:06.763926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:06.788835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539172-a19b-48ea-b901-534cb4cb7be0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:06.805920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:06.808672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:06.809026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:06.809294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:06.812527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:06.815437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:06.866079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 863/1716] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:06.869954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:06.881252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743924d1-8912-4313-83fe-47ba02087176 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:06.898015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:06.898525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:06.898857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:06.899138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] PATH_INFO: `/v3/projects/52bbdd6db22d4079874228c8ee7c982d/users/696b113612944c9486552effbba3c13b/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:06.955346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bbdd6db22d4079874228c8ee7c982d, user_id=696b113612944c9486552effbba3c13b, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:06.958341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:06.960108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:06.970292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:06.986853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14469c1-717f-466b-9617-deb40d72e56d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:06.993814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:06.994492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:06.994861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:06.995149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] PATH_INFO: `/v3/users/341990278d184a298c411bbae2c8474d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:07.001507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=341990278d184a298c411bbae2c8474d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:07.006201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:07.078362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:07.094219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 854/1717] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:06 2022] PUT /identity/v3/projects/52bbdd6db22d4079874228c8ee7c982d/users/696b113612944c9486552effbba3c13b/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:07.099940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:07.277201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '341990278d184a298c411bbae2c8474d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.279731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '341990278d184a298c411bbae2c8474d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.373372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c60287-6f3a-4d7a-b32d-1ffb4c476a22 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:07.385334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:07.386217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:07.386675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:07.387113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:07.388274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:07.390310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:07.396705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '341990278d184a298c411bbae2c8474d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.411173 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 855/1718] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:07.416867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:07.419558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:07.467306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '341990278d184a298c411bbae2c8474d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.487626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-be5626a3-bf6f-4bb1-9cf1-62e1435cd167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '341990278d184a298c411bbae2c8474d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.487713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3aab3-5034-4b64-8563-92106cc80f5e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:07.500742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:07.503155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:07.503737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:07.504445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] PATH_INFO: `/v3/projects/b75be27c6e1445b1b49ba998a9c4a216` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:07.506778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75be27c6e1445b1b49ba998a9c4a216)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:07.509072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:07.509981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 864/1719] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:06 2022] DELETE /identity/v3/users/341990278d184a298c411bbae2c8474d => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:07.511530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:07.512109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:07.512464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:07.600137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] MFA Rules not processed for user `b9252893998a490689497697663ca4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:07.603456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:07.740122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75be27c6e1445b1b49ba998a9c4a216', 'request_id': 'req-7a4bafd0-aef7-4127-8bc2-9fbce3751131'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.765473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:07.767100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75be27c6e1445b1b49ba998a9c4a216', 'request_id': 'req-7a4bafd0-aef7-4127-8bc2-9fbce3751131'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.780612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75be27c6e1445b1b49ba998a9c4a216', 'request_id': 'req-7a4bafd0-aef7-4127-8bc2-9fbce3751131'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.804054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 865/1720] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:07.805572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:07.807561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:07.868670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] The token cache is being invalidate because project b75be27c6e1445b1b49ba998a9c4a216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:07.868821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.872195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a4bafd0-aef7-4127-8bc2-9fbce3751131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:07.875599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 856/1721] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:07 2022] DELETE /identity/v3/projects/b75be27c6e1445b1b49ba998a9c4a216 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:07.876647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b0078-03aa-422d-b5d0-42cdf56de217 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:07.877112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b0078-03aa-422d-b5d0-42cdf56de217 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:07.877215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b0078-03aa-422d-b5d0-42cdf56de217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:07.926898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-847b0078-03aa-422d-b5d0-42cdf56de217 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:07.929399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-847b0078-03aa-422d-b5d0-42cdf56de217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:07.947774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f3fcb-f68e-401f-9c0e-8e792da81a94 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:08.073718 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969d92a-2780-4da9-9c06-9870f7a3acb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:08.074416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969d92a-2780-4da9-9c06-9870f7a3acb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.074731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969d92a-2780-4da9-9c06-9870f7a3acb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.075015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969d92a-2780-4da9-9c06-9870f7a3acb7 admin admin] PATH_INFO: `/v3/projects/52bbdd6db22d4079874228c8ee7c982d/users/696b113612944c9486552effbba3c13b/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.077055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969d92a-2780-4da9-9c06-9870f7a3acb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bbdd6db22d4079874228c8ee7c982d, user_id=696b113612944c9486552effbba3c13b, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:08.080884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969d92a-2780-4da9-9c06-9870f7a3acb7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:08.120090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 866/1722] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:07 2022] PUT /identity/v3/projects/52bbdd6db22d4079874228c8ee7c982d/users/696b113612944c9486552effbba3c13b/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:08.120090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5054-26d4-43b6-bfb3-cece4187de0e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.120090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5054-26d4-43b6-bfb3-cece4187de0e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.120090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5054-26d4-43b6-bfb3-cece4187de0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.130743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 857/1723] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:08.130743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.130743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.130743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.141624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ea3d5054-26d4-43b6-bfb3-cece4187de0e None None] MFA Rules not processed for user `b9252893998a490689497697663ca4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:08.146864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ea3d5054-26d4-43b6-bfb3-cece4187de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.231309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:08.232542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.292223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 867/1724] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:08.293440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.293861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.294272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.398266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 858/1725] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:08.399223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:08.406629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.407314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] MFA Rules not processed for user `696b113612944c9486552effbba3c13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:08.416101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.593760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a1cda-f8fd-4559-b492-1ba6513a757d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:08.600969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139919a-0ebc-47ef-84fc-be2c1c79d00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:08.601707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139919a-0ebc-47ef-84fc-be2c1c79d00e service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.602057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139919a-0ebc-47ef-84fc-be2c1c79d00e service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.602406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139919a-0ebc-47ef-84fc-be2c1c79d00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.613523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 868/1726] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:08.616327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:08.617216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139919a-0ebc-47ef-84fc-be2c1c79d00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:08.619256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139919a-0ebc-47ef-84fc-be2c1c79d00e service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:08.620699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.659462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 859/1727] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:08.660856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2badbe3-6f3a-44de-9757-24ad792499e5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.661146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2badbe3-6f3a-44de-9757-24ad792499e5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.661414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2badbe3-6f3a-44de-9757-24ad792499e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.733376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d2badbe3-6f3a-44de-9757-24ad792499e5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:08.737842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d2badbe3-6f3a-44de-9757-24ad792499e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.774815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 860/1728] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:08.774815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfd416-65b6-4bf1-a817-cf2e194f32d8 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.774815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfd416-65b6-4bf1-a817-cf2e194f32d8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.774815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfd416-65b6-4bf1-a817-cf2e194f32d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.824207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-02bfd416-65b6-4bf1-a817-cf2e194f32d8 None None] MFA Rules not processed for user `281e6cd581c54901b8feaa291db588eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:08.825490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-02bfd416-65b6-4bf1-a817-cf2e194f32d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.857229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f70fe-bece-417c-bc9e-9905349dc63c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:08.872940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:08.877454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:08.877785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:08.878214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:08.883909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.915502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:08.918333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:08.952931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 869/1729] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:08.955771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:08.959606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:08.992324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 861/1730] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:08.993249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfd416-65b6-4bf1-a817-cf2e194f32d8 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.030043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfd416-65b6-4bf1-a817-cf2e194f32d8 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.043610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f97ae3-9058-45d8-9f1c-edec7accba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.044578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97ae3-9058-45d8-9f1c-edec7accba0a admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.044578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97ae3-9058-45d8-9f1c-edec7accba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.044787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f97ae3-9058-45d8-9f1c-edec7accba0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.045458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f97ae3-9058-45d8-9f1c-edec7accba0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:09.045701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f97ae3-9058-45d8-9f1c-edec7accba0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.047336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f97ae3-9058-45d8-9f1c-edec7accba0a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.075804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 862/1731] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:09.075804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3fb29-f45a-44a4-ae7a-04dbb1e80057 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.075804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3fb29-f45a-44a4-ae7a-04dbb1e80057 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.075804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3fb29-f45a-44a4-ae7a-04dbb1e80057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.104711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059423-e7e1-407a-a921-e485b06f5155 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.116166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.118052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.118642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.119829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0fb3fb29-f45a-44a4-ae7a-04dbb1e80057 None None] MFA Rules not processed for user `281e6cd581c54901b8feaa291db588eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:09.120167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.121736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0fb3fb29-f45a-44a4-ae7a-04dbb1e80057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:09.144242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 863/1732] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:09.146566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3fb29-f45a-44a4-ae7a-04dbb1e80057 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.147109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.148830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.177131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 870/1733] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:09.179172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.180845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:09.258049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3fb29-f45a-44a4-ae7a-04dbb1e80057 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.276497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6b4f-f888-4c30-96d6-09aae6d2aab5 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.278220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.279492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.279789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.280151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.280946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.284009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.291821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.292506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.292786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.293060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.310760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.313346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.332104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 871/1734] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:09.445801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:09.449768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.461544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 864/1735] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:09 2022] POST /identity/v3/projects => generated 363 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:09.465777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3065a-49c6-42cc-9f00-ae1e350df47f service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.553926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.554430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:09.568070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.568958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.569389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.569764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.571214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.573648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.687489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 872/1736] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:09 2022] POST /identity/v3/users => generated 282 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:30:09.691551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.699395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045e271-0d9f-409d-8562-c8db4b2c82ce admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.712078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.713527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d80417-d969-494c-a3e6-50456f881b75 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.714370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.714781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.716065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] PATH_INFO: `/v3/users/5242265288ff407892038bbf7f294d71` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.718519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5242265288ff407892038bbf7f294d71)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.728602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.730101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.736744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.736744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.736744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.736744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.743745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.784226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 873/1737] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:09.792426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.823514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea0559-917b-4d34-867c-f1c512aae2ad admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.839646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:09.846675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.846675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.846675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.846675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.847072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.847929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:09.865799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 874/1738] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:09.873862 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:09.900525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be75e1-553d-49a8-8e20-4c87317ec158 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:09.911339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86951679-45f6-40a1-a6ca-fda68efe87e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:09.912176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86951679-45f6-40a1-a6ca-fda68efe87e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:09.912462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86951679-45f6-40a1-a6ca-fda68efe87e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:09.912741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86951679-45f6-40a1-a6ca-fda68efe87e2 admin admin] PATH_INFO: `/v3/projects/b8fb12fa9d0c4b708e2151936d976208/users/59421570c7194e9d8c371f6a24d6d51c/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:09.949158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86951679-45f6-40a1-a6ca-fda68efe87e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8fb12fa9d0c4b708e2151936d976208, user_id=59421570c7194e9d8c371f6a24d6d51c, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:09.951907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86951679-45f6-40a1-a6ca-fda68efe87e2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:10.028484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5242265288ff407892038bbf7f294d71'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.029377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5242265288ff407892038bbf7f294d71'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.030204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5242265288ff407892038bbf7f294d71'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.041796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5242265288ff407892038bbf7f294d71'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.054390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 875/1739] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:09 2022] PUT /identity/v3/projects/b8fb12fa9d0c4b708e2151936d976208/users/59421570c7194e9d8c371f6a24d6d51c/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:30:10.065384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa2b7-f69a-4e28-95ab-f901ab7298af None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.065691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa2b7-f69a-4e28-95ab-f901ab7298af None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.065933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa2b7-f69a-4e28-95ab-f901ab7298af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.079770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-14bbd327-21b3-4832-9898-07ad220d66a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5242265288ff407892038bbf7f294d71'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.102255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-20baa2b7-f69a-4e28-95ab-f901ab7298af None None] MFA Rules not processed for user `59421570c7194e9d8c371f6a24d6d51c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:10.103674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-20baa2b7-f69a-4e28-95ab-f901ab7298af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.172596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 865/1740] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:09 2022] DELETE /identity/v3/users/5242265288ff407892038bbf7f294d71 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:30:10.172596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f7cda-1e4f-4ea8-b3ff-3c57bc75cecd None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.172596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f7cda-1e4f-4ea8-b3ff-3c57bc75cecd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.172596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f7cda-1e4f-4ea8-b3ff-3c57bc75cecd None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.180060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 866/1741] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 12:30:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:10.191451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cf786-d935-4f86-81a7-29af20659ff8 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.191762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cf786-d935-4f86-81a7-29af20659ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.194703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cf786-d935-4f86-81a7-29af20659ff8 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.199392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 867/1742] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 12:30:10 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:10.201804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cf786-d935-4f86-81a7-29af20659ff8 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:10.228783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c94cf786-d935-4f86-81a7-29af20659ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.246744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 876/1743] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:10.252754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af521c5-3a1e-4e7f-b9f5-3026a073646a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.259031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af521c5-3a1e-4e7f-b9f5-3026a073646a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.259031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af521c5-3a1e-4e7f-b9f5-3026a073646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.312734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2af521c5-3a1e-4e7f-b9f5-3026a073646a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:10.313899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2af521c5-3a1e-4e7f-b9f5-3026a073646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.372484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cf786-d935-4f86-81a7-29af20659ff8 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:10.380903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:10.382003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.382498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.382819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] PATH_INFO: `/v3/projects/b0950bef7bcc479998967ea1df55fe9d` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.384003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0950bef7bcc479998967ea1df55fe9d)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:10.385939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:10.472764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 877/1744] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:10.474178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a832-8ec4-4e02-800d-30ff3986b48b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.474473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a832-8ec4-4e02-800d-30ff3986b48b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.475042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a832-8ec4-4e02-800d-30ff3986b48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.551272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-81f4a832-8ec4-4e02-800d-30ff3986b48b None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:10.553498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-81f4a832-8ec4-4e02-800d-30ff3986b48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.560678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0950bef7bcc479998967ea1df55fe9d', 'request_id': 'req-83510828-b921-4a3a-9a4c-6b49e4896b10'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.571648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0950bef7bcc479998967ea1df55fe9d', 'request_id': 'req-83510828-b921-4a3a-9a4c-6b49e4896b10'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.589390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0950bef7bcc479998967ea1df55fe9d', 'request_id': 'req-83510828-b921-4a3a-9a4c-6b49e4896b10'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.668883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 878/1745] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:10.670338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528ae39-788a-4f73-a8e3-837db111d6aa None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.670630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528ae39-788a-4f73-a8e3-837db111d6aa None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.671551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528ae39-788a-4f73-a8e3-837db111d6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.705640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d528ae39-788a-4f73-a8e3-837db111d6aa None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:10.707428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d528ae39-788a-4f73-a8e3-837db111d6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.709597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] The token cache is being invalidate because project b0950bef7bcc479998967ea1df55fe9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:10.712710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.712710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:10.712710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 868/1746] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:10 2022] DELETE /identity/v3/projects/b0950bef7bcc479998967ea1df55fe9d => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:10.713473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:10.718788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.720606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 879/1747] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:10.729163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.730287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.730548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.758999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] MFA Rules not processed for user `6b94f244190e4af089d070ad83806c38`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:10.760871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.844042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83510828-b921-4a3a-9a4c-6b49e4896b10 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:10.850337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:10.851154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.851437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.851713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.852418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:10.854437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:10.856686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 880/1748] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:10.857774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:10.859312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.873703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cf278-1a65-4782-932f-e0fabe127bdb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:10.879515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748af5-15c4-42aa-bc6f-f9aaf58e9ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:10.880899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748af5-15c4-42aa-bc6f-f9aaf58e9ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.881290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748af5-15c4-42aa-bc6f-f9aaf58e9ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.881659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748af5-15c4-42aa-bc6f-f9aaf58e9ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.886913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a0748af5-15c4-42aa-bc6f-f9aaf58e9ff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.901688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748af5-15c4-42aa-bc6f-f9aaf58e9ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:10.903911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748af5-15c4-42aa-bc6f-f9aaf58e9ff4 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:10.914141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 881/1749] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:10.915697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ecbf6-bd32-49aa-8497-196274ae49e4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.916222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ecbf6-bd32-49aa-8497-196274ae49e4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.916716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ecbf6-bd32-49aa-8497-196274ae49e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.939818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-416ecbf6-bd32-49aa-8497-196274ae49e4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:10.942912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-416ecbf6-bd32-49aa-8497-196274ae49e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.950440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 882/1750] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:10.951367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ecbf6-bd32-49aa-8497-196274ae49e4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:10.960944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ecbf6-bd32-49aa-8497-196274ae49e4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:10.967745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a636060-065c-473b-a0e9-60d911ec516b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:10.968362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a636060-065c-473b-a0e9-60d911ec516b service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.968668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a636060-065c-473b-a0e9-60d911ec516b service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.971733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a636060-065c-473b-a0e9-60d911ec516b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:10.979702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a636060-065c-473b-a0e9-60d911ec516b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:10.981641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a636060-065c-473b-a0e9-60d911ec516b service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:10.987660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 869/1751] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/projects => generated 361 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:10.991144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:10.993986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:10.995278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 883/1752] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:10.996650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8951e-8dfe-4885-a841-f5bf9cbba8a8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:10.996939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8951e-8dfe-4885-a841-f5bf9cbba8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:10.997210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8951e-8dfe-4885-a841-f5bf9cbba8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.041687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-96d8951e-8dfe-4885-a841-f5bf9cbba8a8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:11.043596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-96d8951e-8dfe-4885-a841-f5bf9cbba8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:11.141192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4224-71cd-4a10-bfdb-40ac2ed2564b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.156364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.161858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 884/1753] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:11.164893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.170140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.170140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.170345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8951e-8dfe-4885-a841-f5bf9cbba8a8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.175012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:11.181372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8951e-8dfe-4885-a841-f5bf9cbba8a8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.194117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.194937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.195277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.195624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.196709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.204632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.218963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.218963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.227534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:11.241760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 870/1754] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:11.242855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.256080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63ab34-5091-46c1-8b9a-0425047f813a service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.268013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.269251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.269251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.269416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.270000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:11.270484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.272233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.284615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 871/1755] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:11.294003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.328034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b3d5f-2c43-4826-80b2-1351aac4497f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.336327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.336327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.336579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.336802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.337437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.339397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.373109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 885/1756] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:11 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:11.377053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.391529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8092cc-fd98-4aef-aeb7-c6982878b8f0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.400010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.401274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.402257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.402793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.403878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.405801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.441278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 886/1757] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:11.447641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.480489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 872/1758] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:11 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:11.480592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732057c-c88f-4e0f-be54-af8d329f8aba admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.481068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.583900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.584739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.585424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.585424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] PATH_INFO: `/v3/projects/1582beff4c5d46e5ae4726516a44846a/users/6aab2d5826b2467f8ad462d7a4b1e74d/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.609154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1582beff4c5d46e5ae4726516a44846a, user_id=6aab2d5826b2467f8ad462d7a4b1e74d, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.619495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.660094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22502f9-8a21-4715-9543-557d4818383c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.675764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.676426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.677091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.677091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.677590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.679650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.732106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 887/1759] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:11 2022] PUT /identity/v3/projects/1582beff4c5d46e5ae4726516a44846a/users/6aab2d5826b2467f8ad462d7a4b1e74d/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:11.735800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.835302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 873/1760] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:11 2022] POST /identity/v3/users => generated 286 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:30:11.835302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.854215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bdd40-53f6-4fdb-9cf2-66d0bcd4c2ef admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.916446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab23bbf-8a64-441a-bc78-68eef4fa5d2f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.931820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.932456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.932738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.935524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.935524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:11.937748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:11.953223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 888/1761] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:11.958100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:11.972889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90917c19-ea00-4b70-b14e-2ea911b0c157 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:11.987144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3daff-8883-4427-b732-482cb02c52cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:11.990288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3daff-8883-4427-b732-482cb02c52cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:11.990500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3daff-8883-4427-b732-482cb02c52cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:11.991610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3daff-8883-4427-b732-482cb02c52cf admin admin] PATH_INFO: `/v3/projects/1582beff4c5d46e5ae4726516a44846a/users/6aab2d5826b2467f8ad462d7a4b1e74d/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:11.995792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3daff-8883-4427-b732-482cb02c52cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1582beff4c5d46e5ae4726516a44846a, user_id=6aab2d5826b2467f8ad462d7a4b1e74d, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:12.001430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f4442-0dec-4d52-af5f-46a52169057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:12.002067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f4442-0dec-4d52-af5f-46a52169057c admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.002419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f4442-0dec-4d52-af5f-46a52169057c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.002700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f4442-0dec-4d52-af5f-46a52169057c admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.004172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3daff-8883-4427-b732-482cb02c52cf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:12.005534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f4442-0dec-4d52-af5f-46a52169057c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:12.012433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f4442-0dec-4d52-af5f-46a52169057c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:12.081313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 889/1762] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:11 2022] PUT /identity/v3/projects/1582beff4c5d46e5ae4726516a44846a/users/6aab2d5826b2467f8ad462d7a4b1e74d/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:12.090760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a20feb-28aa-4c6a-aba3-420f4be5a992 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.090979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a20feb-28aa-4c6a-aba3-420f4be5a992 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.091264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a20feb-28aa-4c6a-aba3-420f4be5a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.096453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 874/1763] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:12.097971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.098329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.098715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.162421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f0a20feb-28aa-4c6a-aba3-420f4be5a992 None None] MFA Rules not processed for user `751f05f1ff6c4593870073d8f47e0db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:12.163513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] MFA Rules not processed for user `6aab2d5826b2467f8ad462d7a4b1e74d`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:12.164483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f0a20feb-28aa-4c6a-aba3-420f4be5a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:12.164978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:12.314674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 890/1764] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:12.315568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a20feb-28aa-4c6a-aba3-420f4be5a992 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:12.361094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 875/1765] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:12.366941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:12.368514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:12.412798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a20feb-28aa-4c6a-aba3-420f4be5a992 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:12.449989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a65b5b-8b2f-4f44-9179-facb05d209bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:12.451039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65b5b-8b2f-4f44-9179-facb05d209bb admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.451325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65b5b-8b2f-4f44-9179-facb05d209bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.451646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65b5b-8b2f-4f44-9179-facb05d209bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.452305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a65b5b-8b2f-4f44-9179-facb05d209bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:12.458180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a65b5b-8b2f-4f44-9179-facb05d209bb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:12.495389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 891/1766] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:12.497524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8daca8-aa1e-444e-a0e2-6243f2a9b77e None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.497811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8daca8-aa1e-444e-a0e2-6243f2a9b77e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.498083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8daca8-aa1e-444e-a0e2-6243f2a9b77e None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.499780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 892/1767] 10.222.0.45 () {54 vars in 961 bytes} [Wed Aug 24 12:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:12.500615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8daca8-aa1e-444e-a0e2-6243f2a9b77e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:12.503916 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5b8daca8-aa1e-444e-a0e2-6243f2a9b77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:12.556710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31a70-0132-4a9c-b25a-4ddd528c05f1 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:12.564376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:12.566272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:12.569604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.570429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.570879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] PATH_INFO: `/v3/projects/70ef0e2e49654a40ad801587b8427b00` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.575780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70ef0e2e49654a40ad801587b8427b00)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:12.578033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:12.683092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8daca8-aa1e-444e-a0e2-6243f2a9b77e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:12.687925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:12.689150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.689687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.690224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.736349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:12.736349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:12.743884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 893/1768] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:12.744892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:12.765154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2fb10-7223-49aa-910f-70e2a6fd4702 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:12.791114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70ef0e2e49654a40ad801587b8427b00', 'request_id': 'req-4de2714d-caed-4a39-ab02-e4558b802f77'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:12.801407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70ef0e2e49654a40ad801587b8427b00', 'request_id': 'req-4de2714d-caed-4a39-ab02-e4558b802f77'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:12.815545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70ef0e2e49654a40ad801587b8427b00', 'request_id': 'req-4de2714d-caed-4a39-ab02-e4558b802f77'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:12.909808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] The token cache is being invalidate because project 70ef0e2e49654a40ad801587b8427b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:12.914340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:12.914340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4de2714d-caed-4a39-ab02-e4558b802f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:12.920056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 876/1769] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:12 2022] DELETE /identity/v3/projects/70ef0e2e49654a40ad801587b8427b00 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:12.920056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766a14a-82bd-4a74-9cc1-6be8ae717979 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.920056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766a14a-82bd-4a74-9cc1-6be8ae717979 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.920056 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766a14a-82bd-4a74-9cc1-6be8ae717979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.924719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:12.930302 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:12.931211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:12.931639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] PATH_INFO: `/v3/projects/9189324d1b0f4f2d99fedbb18bde61e9/users/ceba49bce923449ca7a04727965ce796/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:12.997558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1766a14a-82bd-4a74-9cc1-6be8ae717979 None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:12.998286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1766a14a-82bd-4a74-9cc1-6be8ae717979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:13.022989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9189324d1b0f4f2d99fedbb18bde61e9, user_id=ceba49bce923449ca7a04727965ce796, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:13.029652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:13.140055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 894/1770] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:12 2022] PUT /identity/v3/projects/9189324d1b0f4f2d99fedbb18bde61e9/users/ceba49bce923449ca7a04727965ce796/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:13.140055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:13.149951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:13.150164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 877/1771] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Aug 24 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:13.158875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:13.159422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:13.159873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:13.184507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] MFA Rules not processed for user `b9252893998a490689497697663ca4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:13.191019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:13.273474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 878/1772] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:13.279128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:13.279128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:13.412463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2a197-1f29-4e97-85ca-835df06bbcc5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:13.428939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2a09b-910f-4d81-9e7c-82cbbc459c9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:13.429563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a09b-910f-4d81-9e7c-82cbbc459c9d service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:13.429838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a09b-910f-4d81-9e7c-82cbbc459c9d service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:13.430206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a09b-910f-4d81-9e7c-82cbbc459c9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:13.435466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fba2a09b-910f-4d81-9e7c-82cbbc459c9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:13.459059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2a09b-910f-4d81-9e7c-82cbbc459c9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:13.459849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca3a40-39ef-4746-815c-be887f87069c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:13.461321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2a09b-910f-4d81-9e7c-82cbbc459c9d service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:13.484516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:13.485952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:13.487908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:13.492576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] PATH_INFO: `/v3/users/54bef09a96724d14a8e091110d958bec` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:13.496528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 895/1773] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:13.499758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=54bef09a96724d14a8e091110d958bec)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:13.500238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d8a60-b3a5-49e2-8527-47b20a8533d8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:13.500553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d8a60-b3a5-49e2-8527-47b20a8533d8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:13.503290 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:13.508444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d8a60-b3a5-49e2-8527-47b20a8533d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:13.593009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ae1d8a60-b3a5-49e2-8527-47b20a8533d8 None None] MFA Rules not processed for user `ceba49bce923449ca7a04727965ce796`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:13.598508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ae1d8a60-b3a5-49e2-8527-47b20a8533d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:13.746299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 896/1774] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:13.746299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d8a60-b3a5-49e2-8527-47b20a8533d8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:13.776513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d8a60-b3a5-49e2-8527-47b20a8533d8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:13.794392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:13.794392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:13.794392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:13.794392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:13.812753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:13.817716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:13.839341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:13.845155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 897/1775] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:13.853589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:13.863170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54bef09a96724d14a8e091110d958bec'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:13.866975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54bef09a96724d14a8e091110d958bec'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:13.871201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54bef09a96724d14a8e091110d958bec'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:13.871720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:13.892519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54bef09a96724d14a8e091110d958bec'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:13.930390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54bef09a96724d14a8e091110d958bec'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:13.946721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88167d-3ee1-4b21-8c83-e000234cfded service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:13.960846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:13.961476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:13.961758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:13.962030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:13.962682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:13.965051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:14.017529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 879/1776] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:13 2022] DELETE /identity/v3/users/54bef09a96724d14a8e091110d958bec => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:14.080137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:14.128359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 898/1777] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:13 2022] POST /identity/v3/projects => generated 369 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:14.132908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:14.134558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:14.335827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cb4be-9f7c-40a2-91c9-4dea961497bc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:14.419400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:14.421261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:14.583063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a105b-41c0-4941-9362-25a87fd77060 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:14.585872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:14.586534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:14.586824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:14.587101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:14.587748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:14.589682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:14.601126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:14.602021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:14.602472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:14.602906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] PATH_INFO: `/v3/users/64a76dd851f44de48681960461c70d8e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:14.604713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a76dd851f44de48681960461c70d8e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:14.606855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:14.721458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 880/1778] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:14 2022] POST /identity/v3/users => generated 291 bytes in 589 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:14.723849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:14.730627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:14.758295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc77b5a-3004-470d-9f8e-5206dfcd4925 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:14.778899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:14.779815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:14.783993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:14.783993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] PATH_INFO: `/v3/users/59421570c7194e9d8c371f6a24d6d51c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:14.785253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=59421570c7194e9d8c371f6a24d6d51c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:14.788320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:14.864560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:14.913473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a76dd851f44de48681960461c70d8e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:14.915584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a76dd851f44de48681960461c70d8e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.048060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a76dd851f44de48681960461c70d8e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.089263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a76dd851f44de48681960461c70d8e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.090313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a76dd851f44de48681960461c70d8e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.130842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 899/1779] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:14 2022] DELETE /identity/v3/users/64a76dd851f44de48681960461c70d8e => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:15.130842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:15.130842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:15.133041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59421570c7194e9d8c371f6a24d6d51c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.137656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59421570c7194e9d8c371f6a24d6d51c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.137656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59421570c7194e9d8c371f6a24d6d51c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.155342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59421570c7194e9d8c371f6a24d6d51c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.207634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-0295947c-1eec-422f-a538-78a93281b0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59421570c7194e9d8c371f6a24d6d51c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:15.343956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 881/1780] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:14 2022] DELETE /identity/v3/users/59421570c7194e9d8c371f6a24d6d51c => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:15.350630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:15.352639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:15.356553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:15.467770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa342b-e467-4861-a394-8750d5cfd3c3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:15.472326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:15.476043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:15.645052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2a126-1c63-4f76-8703-073f5cdde782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:15.649084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a126-1c63-4f76-8703-073f5cdde782 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:15.649538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a126-1c63-4f76-8703-073f5cdde782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:15.650535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a126-1c63-4f76-8703-073f5cdde782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:15.652169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2a126-1c63-4f76-8703-073f5cdde782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:15.657903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 882/1781] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:15.658878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:15.660606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:15.661950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2a126-1c63-4f76-8703-073f5cdde782 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:15.681836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1f58e-848a-402e-b830-4d202fcfad2c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:15.687665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:15.690977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:15.691438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:15.704772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] PATH_INFO: `/v3/projects/2a8300a4efa049638d148d246a7086f7` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:15.708689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8300a4efa049638d148d246a7086f7)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:15.712485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:15.722540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 900/1782] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:15.724322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:15.731664 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:15.732422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:15.795287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:15.796697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:15.842072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 901/1783] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:15.848548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:15.851464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:15.957255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d42f9-c38f-44bd-9e13-a79b0578ecd0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:15.966900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:15.967515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:15.967790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:15.968060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:15.988185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:15.991015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:16.061451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 902/1784] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:16.063340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:16.075887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:16.078409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8300a4efa049638d148d246a7086f7', 'request_id': 'req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:16.086340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8300a4efa049638d148d246a7086f7', 'request_id': 'req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:16.093075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8300a4efa049638d148d246a7086f7', 'request_id': 'req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:16.094206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5616d-f999-4d53-af74-254aa5e55490 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:16.235318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] The token cache is being invalidate because project 2a8300a4efa049638d148d246a7086f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:16.235934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:16.236917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:16.240658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 883/1785] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:15 2022] DELETE /identity/v3/projects/2a8300a4efa049638d148d246a7086f7 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:16.241195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:16.243166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:16.272195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:16.276102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:16.276102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:16.276102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] PATH_INFO: `/v3/projects/c45727e22bf5496fb71f0dfa2e156d95/users/6c3a7b17d53e4d1e9d2275738925a535/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:16.343792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:16.372499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45727e22bf5496fb71f0dfa2e156d95, user_id=6c3a7b17d53e4d1e9d2275738925a535, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:16.391868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:16.449538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4c6e5-fbfa-4d07-a35e-dccc0e4baafc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:16.469787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465178d7-a173-4cb3-9348-ba8b69fd03ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:16.469787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465178d7-a173-4cb3-9348-ba8b69fd03ce service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:16.469787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465178d7-a173-4cb3-9348-ba8b69fd03ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:16.469787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465178d7-a173-4cb3-9348-ba8b69fd03ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:16.470375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-465178d7-a173-4cb3-9348-ba8b69fd03ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:16.472141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465178d7-a173-4cb3-9348-ba8b69fd03ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:16.474603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465178d7-a173-4cb3-9348-ba8b69fd03ce service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:16.497726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 884/1786] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:16.498141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63192a5-7903-492a-ad26-a95b1b520291 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:16.500491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63192a5-7903-492a-ad26-a95b1b520291 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:16.500888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63192a5-7903-492a-ad26-a95b1b520291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:16.533479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 903/1787] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:16 2022] PUT /identity/v3/projects/c45727e22bf5496fb71f0dfa2e156d95/users/6c3a7b17d53e4d1e9d2275738925a535/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:16.533479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:16.536711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:16.637102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a63192a5-7903-492a-ad26-a95b1b520291 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:16.682735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a63192a5-7903-492a-ad26-a95b1b520291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:16.774304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530fbab-956d-4933-b564-5f536e0588fa admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:16.809101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:16.813785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:16.814193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:16.814311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:16.814975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:16.817453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:16.830362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:16.891392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 904/1788] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:16.899506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:16.942468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc7add-f8a3-4b86-8c9b-701c252a701a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:16.964288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf2b74-43c4-41cc-8b86-985fb313aa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:16.964924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf2b74-43c4-41cc-8b86-985fb313aa48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:16.965171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf2b74-43c4-41cc-8b86-985fb313aa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:16.965490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf2b74-43c4-41cc-8b86-985fb313aa48 admin admin] PATH_INFO: `/v3/projects/c45727e22bf5496fb71f0dfa2e156d95/users/6c3a7b17d53e4d1e9d2275738925a535/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:16.968037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 885/1789] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:16.969125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf2b74-43c4-41cc-8b86-985fb313aa48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45727e22bf5496fb71f0dfa2e156d95, user_id=6c3a7b17d53e4d1e9d2275738925a535, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:16.977115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def18c30-0fe1-4d30-9bfd-82063fed9626 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:16.977393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def18c30-0fe1-4d30-9bfd-82063fed9626 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:16.977507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf2b74-43c4-41cc-8b86-985fb313aa48 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:16.977713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def18c30-0fe1-4d30-9bfd-82063fed9626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:17.083922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 905/1790] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:16 2022] PUT /identity/v3/projects/c45727e22bf5496fb71f0dfa2e156d95/users/6c3a7b17d53e4d1e9d2275738925a535/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:30:17.084445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-def18c30-0fe1-4d30-9bfd-82063fed9626 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:17.084445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-def18c30-0fe1-4d30-9bfd-82063fed9626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:17.102104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f5db4-fbb8-4061-baa6-7b0221ae6cc3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:17.102712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f5db4-fbb8-4061-baa6-7b0221ae6cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:17.103928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f5db4-fbb8-4061-baa6-7b0221ae6cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:17.158949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5d3f5db4-fbb8-4061-baa6-7b0221ae6cc3 None None] MFA Rules not processed for user `6c3a7b17d53e4d1e9d2275738925a535`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:17.160197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5d3f5db4-fbb8-4061-baa6-7b0221ae6cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:17.170950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 886/1791] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:17.179947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def18c30-0fe1-4d30-9bfd-82063fed9626 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:17.208198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def18c30-0fe1-4d30-9bfd-82063fed9626 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:17.221558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:17.237855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:17.238149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:17.238423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:17.241278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:17.241575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:17.243273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:17.275525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 887/1792] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:17.276443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:17.282273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:17.320956 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5cc06-24be-4225-aa1e-94848f065207 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:17.331600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:17.334848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:17.334848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:17.334848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] PATH_INFO: `/v3/projects/5a1327efac234cf9ba49a4fd1285305c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:17.336170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1327efac234cf9ba49a4fd1285305c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:17.341086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:17.447366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 906/1793] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:17.451837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f5db4-fbb8-4061-baa6-7b0221ae6cc3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:17.466286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f5db4-fbb8-4061-baa6-7b0221ae6cc3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:17.501965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:17.501965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:17.501965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:17.501965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:17.502443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:17.503997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:17.642373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1327efac234cf9ba49a4fd1285305c', 'request_id': 'req-c4761b14-0108-4525-9901-7552e6edc40c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:17.664680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1327efac234cf9ba49a4fd1285305c', 'request_id': 'req-c4761b14-0108-4525-9901-7552e6edc40c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:17.673601 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1327efac234cf9ba49a4fd1285305c', 'request_id': 'req-c4761b14-0108-4525-9901-7552e6edc40c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:17.685558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 907/1794] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:17 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:17.686433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:17.689368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:17.802006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] The token cache is being invalidate because project 5a1327efac234cf9ba49a4fd1285305c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:17.802006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:17.805025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:17.810608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 888/1795] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:17 2022] DELETE /identity/v3/projects/5a1327efac234cf9ba49a4fd1285305c => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:17.811726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:17.815820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea768317-339e-47d7-adf2-16715c1a8655 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:17.816808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:17.822434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:17.824397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:17.828927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:17.828927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:17.828927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:17.949028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4761b14-0108-4525-9901-7552e6edc40c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:17.967829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:17.970303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:17.985139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 908/1796] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:17.988307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:17.990333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:18.028944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.029740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.030583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.031081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.033234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.037526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.109472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf87fa5-99eb-413d-8137-d4e7de80e84f service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:18.119854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.120509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.120836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.121113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] PATH_INFO: `/v3/projects/b8fb12fa9d0c4b708e2151936d976208` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.122257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8fb12fa9d0c4b708e2151936d976208)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.124405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.168405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 889/1797] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:17 2022] POST /identity/v3/users => generated 294 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:18.169522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:18.172036 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:18.193584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c4c9a-dcf6-4fde-924d-2473e1fef310 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:18.210814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.212754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.213071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.213449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] PATH_INFO: `/v3/users/696b113612944c9486552effbba3c13b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.215384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696b113612944c9486552effbba3c13b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.217812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.290542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:18.302933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8fb12fa9d0c4b708e2151936d976208', 'request_id': 'req-9c6b2a64-1831-4d97-b08c-d3e872948567'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.309549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8fb12fa9d0c4b708e2151936d976208', 'request_id': 'req-9c6b2a64-1831-4d97-b08c-d3e872948567'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.317462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8fb12fa9d0c4b708e2151936d976208', 'request_id': 'req-9c6b2a64-1831-4d97-b08c-d3e872948567'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.419793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] The token cache is being invalidate because project b8fb12fa9d0c4b708e2151936d976208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:18.420085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.421355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.423143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 909/1798] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:17 2022] DELETE /identity/v3/projects/b8fb12fa9d0c4b708e2151936d976208 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:18.427030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:18.428522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:18.458761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696b113612944c9486552effbba3c13b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.459657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696b113612944c9486552effbba3c13b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.460867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696b113612944c9486552effbba3c13b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.468311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696b113612944c9486552effbba3c13b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.480875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696b113612944c9486552effbba3c13b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.506769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b2a64-1831-4d97-b08c-d3e872948567 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:18.509097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.509229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.509464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.509733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.510366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.511998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.525844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 910/1799] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:18.526731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:18.528720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:18.597680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 890/1800] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:18 2022] DELETE /identity/v3/users/696b113612944c9486552effbba3c13b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:18.601962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:18.604509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:18.610622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:18.618920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c26dd-30d0-4385-9cd3-068c5149ac69 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:18.720602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c17466-920e-4902-8a7b-1714ba4805b7 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:18.737045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.737953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.738396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.738966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] PATH_INFO: `/v3/users/6aab2d5826b2467f8ad462d7a4b1e74d` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.741000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aab2d5826b2467f8ad462d7a4b1e74d)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.743216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.795049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.795049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.795049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.795049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.798201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:18.877954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.880312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.889683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 911/1801] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:18.890479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:18.891893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:18.907266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218e4b-9bb9-4a90-a1ff-12e798cf54a0 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:18.913373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.914058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.914505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.914822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.916388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.918488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.930787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 912/1802] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:18.935261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:18.948360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32833a30-a6cf-4d6c-9b29-48d8174d4959 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:18.954609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743b353-eee8-4973-9313-f963215568a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:18.955426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743b353-eee8-4973-9313-f963215568a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:18.956214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743b353-eee8-4973-9313-f963215568a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:18.956662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743b353-eee8-4973-9313-f963215568a4 admin admin] PATH_INFO: `/v3/projects/0871f3f2a71d43d3b96c3a9c797a358c/users/5a22796b7b654e58840b993feed4ed79/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:18.975257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743b353-eee8-4973-9313-f963215568a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0871f3f2a71d43d3b96c3a9c797a358c, user_id=5a22796b7b654e58840b993feed4ed79, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:18.977739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743b353-eee8-4973-9313-f963215568a4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:18.998054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aab2d5826b2467f8ad462d7a4b1e74d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.998909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aab2d5826b2467f8ad462d7a4b1e74d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:18.999643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aab2d5826b2467f8ad462d7a4b1e74d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:19.007759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aab2d5826b2467f8ad462d7a4b1e74d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:19.021532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aab2d5826b2467f8ad462d7a4b1e74d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:19.072554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 913/1803] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:18 2022] PUT /identity/v3/projects/0871f3f2a71d43d3b96c3a9c797a358c/users/5a22796b7b654e58840b993feed4ed79/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:19.083928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311f7e-be6c-4ca4-9739-b002aa8e272f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:19.084338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311f7e-be6c-4ca4-9739-b002aa8e272f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:19.084642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311f7e-be6c-4ca4-9739-b002aa8e272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:19.117050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7f311f7e-be6c-4ca4-9739-b002aa8e272f None None] MFA Rules not processed for user `5a22796b7b654e58840b993feed4ed79`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:19.119085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7f311f7e-be6c-4ca4-9739-b002aa8e272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:19.125328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 891/1804] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:18 2022] DELETE /identity/v3/users/6aab2d5826b2467f8ad462d7a4b1e74d => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:19.221000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 914/1805] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:19.227782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:19.229853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:19.358136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a574b-5b02-461c-8473-24512d86ac68 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:19.374427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:19.375225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:19.375675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:19.376010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:19.376764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:19.383026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:19.496333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 892/1806] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:19 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:19.502327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f311f7e-be6c-4ca4-9739-b002aa8e272f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:19.534172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:19.540911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:19.672603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f311f7e-be6c-4ca4-9739-b002aa8e272f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:19.680334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7efaa2-f1ba-4505-b1a8-e4b480503119 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:19.684632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:19.688020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:19.688354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:19.688613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:19.690083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:19.691922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:19.692769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:19.693258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:19.694008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] PATH_INFO: `/v3/projects/52bbdd6db22d4079874228c8ee7c982d` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:19.694230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:19.695818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52bbdd6db22d4079874228c8ee7c982d)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:19.703871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:19.818996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:19.830237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 915/1807] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:19 2022] POST /identity/v3/users => generated 300 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:19.830237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:19.860484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091bdf2-7fc0-4e1f-a155-6bf3e0a7a5a8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:19.878578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:19.888185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:19.893213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:19.893213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:19.893213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:19.902135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:19.920583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 916/1808] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:19.928532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:19.935536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52bbdd6db22d4079874228c8ee7c982d', 'request_id': 'req-a42ca11a-8a99-4ab2-91df-406af0d81b79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:19.947629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52bbdd6db22d4079874228c8ee7c982d', 'request_id': 'req-a42ca11a-8a99-4ab2-91df-406af0d81b79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:19.950679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447d65-eb92-489c-bd24-c127174dca2a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:19.956060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52bbdd6db22d4079874228c8ee7c982d', 'request_id': 'req-a42ca11a-8a99-4ab2-91df-406af0d81b79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.061271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313252-a5b2-4152-a708-0cf52fa0bf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:20.061271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313252-a5b2-4152-a708-0cf52fa0bf18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.061271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313252-a5b2-4152-a708-0cf52fa0bf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.062452 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313252-a5b2-4152-a708-0cf52fa0bf18 admin admin] PATH_INFO: `/v3/projects/51eb54a16bc74d1c9ec55bb093932920/users/cddd95c4e6984e82baa7c954e13dec76/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:20.091759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] The token cache is being invalidate because project 52bbdd6db22d4079874228c8ee7c982d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:20.092297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.094901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a42ca11a-8a99-4ab2-91df-406af0d81b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.098041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 893/1809] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:19 2022] DELETE /identity/v3/projects/52bbdd6db22d4079874228c8ee7c982d => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:20.104929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.105215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.107026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:20.114470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313252-a5b2-4152-a708-0cf52fa0bf18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51eb54a16bc74d1c9ec55bb093932920, user_id=cddd95c4e6984e82baa7c954e13dec76, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:20.123649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313252-a5b2-4152-a708-0cf52fa0bf18 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:20.182364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:20.183788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:20.211999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 894/1810] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:20.213810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:20.217987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:20.233670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 917/1811] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:19 2022] PUT /identity/v3/projects/51eb54a16bc74d1c9ec55bb093932920/users/cddd95c4e6984e82baa7c954e13dec76/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:20.236611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.236611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.236611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:20.264863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e2b6-719d-4ece-87f6-70382d000d1e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:20.284933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:20.286214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:20.407645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:20.463968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:20.464636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.464915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.465191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] PATH_INFO: `/v3/projects/1582beff4c5d46e5ae4726516a44846a` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:20.473581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1582beff4c5d46e5ae4726516a44846a)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:20.475574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:20.477413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 918/1812] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:20.478375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:20.482533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:20.519654 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b81ae-abc4-494e-9f33-3b1a0b2c9b26 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:20.531707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:20.532347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.532629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.532902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:20.536240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:20.537902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:20.564138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 919/1813] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:20.564138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:20.577674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b964a7-eac3-4d2b-89bd-c0f9f24847b6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:20.583929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:20.584553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.584833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.585109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:20.585877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:20.586235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:20.588049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:20.623382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 920/1814] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:20.627641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:20.680923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1582beff4c5d46e5ae4726516a44846a', 'request_id': 'req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.686372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc850ad-b8e8-40c6-a167-842b474bed92 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:20.689744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1582beff4c5d46e5ae4726516a44846a', 'request_id': 'req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.714704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1582beff4c5d46e5ae4726516a44846a', 'request_id': 'req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.822966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] The token cache is being invalidate because project 1582beff4c5d46e5ae4726516a44846a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:20.823338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.824085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:20.827053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 895/1815] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:20 2022] DELETE /identity/v3/projects/1582beff4c5d46e5ae4726516a44846a => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:20.827961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:20.830843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:20.865324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e8433-79f3-4c80-82b6-c3fa068573ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:20.870512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e8433-79f3-4c80-82b6-c3fa068573ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.870512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e8433-79f3-4c80-82b6-c3fa068573ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.870512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e8433-79f3-4c80-82b6-c3fa068573ea admin admin] PATH_INFO: `/v3/projects/51eb54a16bc74d1c9ec55bb093932920/users/cddd95c4e6984e82baa7c954e13dec76/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:20.872564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e8433-79f3-4c80-82b6-c3fa068573ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=51eb54a16bc74d1c9ec55bb093932920, user_id=cddd95c4e6984e82baa7c954e13dec76, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:20.876514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e8433-79f3-4c80-82b6-c3fa068573ea admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:20.908820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 921/1816] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:20 2022] PUT /identity/v3/projects/51eb54a16bc74d1c9ec55bb093932920/users/cddd95c4e6984e82baa7c954e13dec76/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:20.930895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3c1b-9fc5-49a1-80bf-476ddae64c50 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:20.930895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3c1b-9fc5-49a1-80bf-476ddae64c50 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:20.931215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3c1b-9fc5-49a1-80bf-476ddae64c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:21.039647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20af7e-d4b9-4a86-88bf-3366ec7fc313 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:21.129203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-322e3c1b-9fc5-49a1-80bf-476ddae64c50 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:21.136051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-322e3c1b-9fc5-49a1-80bf-476ddae64c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:21.161822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1b357-a798-4b29-95b0-27a985b987e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:21.163433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1b357-a798-4b29-95b0-27a985b987e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:21.163935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1b357-a798-4b29-95b0-27a985b987e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:21.164441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1b357-a798-4b29-95b0-27a985b987e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:21.165845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1b357-a798-4b29-95b0-27a985b987e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:21.170327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1b357-a798-4b29-95b0-27a985b987e5 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:21.214097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:21.274568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 896/1817] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:20 2022] POST /identity/v3/projects => generated 375 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:21.295771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fd81b-c1ba-4bda-b674-b961cb60682e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:21.296072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fd81b-c1ba-4bda-b674-b961cb60682e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:21.296338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fd81b-c1ba-4bda-b674-b961cb60682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:21.355937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cd5fd81b-c1ba-4bda-b674-b961cb60682e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:21.359519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 922/1818] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:21.360609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7f64a-6191-438f-a2c6-465816e958a3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:21.360890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7f64a-6191-438f-a2c6-465816e958a3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:21.361159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7f64a-6191-438f-a2c6-465816e958a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:21.362674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cd5fd81b-c1ba-4bda-b674-b961cb60682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:21.433485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2aa7f64a-6191-438f-a2c6-465816e958a3 None None] MFA Rules not processed for user `cddd95c4e6984e82baa7c954e13dec76`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:21.436026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2aa7f64a-6191-438f-a2c6-465816e958a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:21.561426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 897/1819] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:21.571357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fd81b-c1ba-4bda-b674-b961cb60682e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:21.598722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fd81b-c1ba-4bda-b674-b961cb60682e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:21.615542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:21.616210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:21.616494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:21.616770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:21.619312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:21.624051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:21.675400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:21.732060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 923/1820] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:21.736650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e8ca3-aee9-4450-ad4d-923b591b274e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:21.737409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e8ca3-aee9-4450-ad4d-923b591b274e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:21.737409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e8ca3-aee9-4450-ad4d-923b591b274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:21.763512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 898/1821] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:21 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:21.765127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:21.776059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:21.796829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1f2e8ca3-aee9-4450-ad4d-923b591b274e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:21.801993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1f2e8ca3-aee9-4450-ad4d-923b591b274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:21.839035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 924/1822] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:21.840098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e8ca3-aee9-4450-ad4d-923b591b274e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:21.881178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e8ca3-aee9-4450-ad4d-923b591b274e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:21.903834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:21.905614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:21.905923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:21.906203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:21.906922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:21.914601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:21.973248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 925/1823] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:21.973846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff11f52-2cbc-4fa9-92d8-764f2ad4a0ea admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:21.975624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:21.986266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:21.987187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:21.989323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:21.989974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.010442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:22.020472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.022112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.054752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 899/1824] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:22.059165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:22.088820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec5d4b-7278-4b61-b891-a87099b520e7 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:22.103714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:22.104520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:22.104921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:22.106633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.107592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:22.110370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.112202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.135119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 900/1825] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:22.136410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:22.170967 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84f70-389a-470c-9a6d-5db7c79498c2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:22.181382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:22.184894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:22.185565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:22.185906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:22.186265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.187034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.188752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.198740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2597205-58fd-4e39-a563-45ab0383e06e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:22.202254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 901/1826] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:22.203069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:22.209860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:22.211981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:22.213314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:22.213628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.215630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fd749-ea42-418d-9ecf-3e6e83a6faee admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:22.223817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:22.223817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:22.223817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:22.223817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.225950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.236988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.337033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.338832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.355720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 926/1827] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:30:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:30:22.356538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:22.370779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d46d6-ebe6-4a16-b641-c04412b61d6e service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:22.378123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 902/1828] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:22 2022] POST /identity/v3/projects => generated 407 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:22.378836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:22.380667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:22.490063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:22.490887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:22.491185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:22.493753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] PATH_INFO: `/v3/projects/0450a0f589a74fcc82f8335c4a699db0/users/9d9cc1c457d1419b9ef2f7d7b723ef37/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.513600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db19d4-0655-4560-a6b3-9a381dfc36db admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:22.519773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0450a0f589a74fcc82f8335c4a699db0, user_id=9d9cc1c457d1419b9ef2f7d7b723ef37, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.523720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.529186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:22.530361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:22.530810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:22.531224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] PATH_INFO: `/v3/users/5a22796b7b654e58840b993feed4ed79` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.534708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a22796b7b654e58840b993feed4ed79)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.539515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.604265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 927/1829] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:22 2022] PUT /identity/v3/projects/0450a0f589a74fcc82f8335c4a699db0/users/9d9cc1c457d1419b9ef2f7d7b723ef37/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:22.609445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:22.776670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9295c-d636-45b6-a434-79a1f5f35076 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:22.809909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a22796b7b654e58840b993feed4ed79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:22.809909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a22796b7b654e58840b993feed4ed79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:22.809909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a22796b7b654e58840b993feed4ed79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:22.809909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a22796b7b654e58840b993feed4ed79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:22.810507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d268c9-1869-4083-8d21-119eadea1e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:22.810507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d268c9-1869-4083-8d21-119eadea1e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:22.810507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d268c9-1869-4083-8d21-119eadea1e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:22.810507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d268c9-1869-4083-8d21-119eadea1e07 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:22.810843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d268c9-1869-4083-8d21-119eadea1e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:22.810843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d268c9-1869-4083-8d21-119eadea1e07 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:22.828183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a22796b7b654e58840b993feed4ed79'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:23.029094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 928/1830] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:22 2022] POST /identity/v3/users => generated 304 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:23.031093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89ece-643c-4891-93d9-2f41ef7c83e1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:23.031093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89ece-643c-4891-93d9-2f41ef7c83e1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:23.031093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89ece-643c-4891-93d9-2f41ef7c83e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:23.033069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 903/1831] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:22 2022] DELETE /identity/v3/users/5a22796b7b654e58840b993feed4ed79 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:23.034259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:23.039024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:23.142494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-28c89ece-643c-4891-93d9-2f41ef7c83e1 None None] MFA Rules not processed for user `9d9cc1c457d1419b9ef2f7d7b723ef37`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:23.143830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-28c89ece-643c-4891-93d9-2f41ef7c83e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:23.220976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc7d05-10e1-4234-a3b8-1355c58dd001 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:23.235239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebd7db-ef26-43a1-ba1e-fd1274370158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:23.236977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd7db-ef26-43a1-ba1e-fd1274370158 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:23.237480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd7db-ef26-43a1-ba1e-fd1274370158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:23.238009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd7db-ef26-43a1-ba1e-fd1274370158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:23.239223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebd7db-ef26-43a1-ba1e-fd1274370158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:23.244217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebd7db-ef26-43a1-ba1e-fd1274370158 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:23.269007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 904/1832] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:23.274610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:23.275375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:23.276009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:23.335241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 929/1833] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:23.340305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c89ece-643c-4891-93d9-2f41ef7c83e1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:23.346041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:23.351732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:23.374774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 905/1834] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:23.376869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:23.382314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:23.395748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c89ece-643c-4891-93d9-2f41ef7c83e1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:23.409057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6437d-5c47-4498-be27-f431675c1172 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:23.409974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:23.410930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:23.411378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:23.411805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:23.412918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:23.415101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:23.443795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:23.447112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:23.447827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:23.448165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:23.448633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:23.450071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:23.454204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:23.480481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 930/1835] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:23.481675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:23.531905 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24865f-4ebb-4907-977e-d7f4fd00c2c7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:23.546478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:23.547671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:23.548443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:23.549337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] PATH_INFO: `/v3/projects/acf75b4993bb4dbd9dfa3630691fde9f/users/eb13886f8b7040c1846bd68bd946745e/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:23.601888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf75b4993bb4dbd9dfa3630691fde9f, user_id=eb13886f8b7040c1846bd68bd946745e, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:23.602922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 906/1836] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:23 2022] POST /identity/v3/projects => generated 377 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:23.606770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:23.608325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:23.612721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:23.759185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 931/1837] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:23 2022] PUT /identity/v3/projects/acf75b4993bb4dbd9dfa3630691fde9f/users/eb13886f8b7040c1846bd68bd946745e/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:23.760566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:23.807994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5af88-19dc-4157-8f23-014ea992a5d2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:23.958326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f51b0-9317-4dda-ad75-3218a1f7fd59 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:23.974772 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7e3fa-deb5-4b09-8783-70355745782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:23.978531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7e3fa-deb5-4b09-8783-70355745782a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:23.979127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7e3fa-deb5-4b09-8783-70355745782a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:23.979408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7e3fa-deb5-4b09-8783-70355745782a admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:23.980280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7e3fa-deb5-4b09-8783-70355745782a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:23.983967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7e3fa-deb5-4b09-8783-70355745782a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:24.037242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867f05b-26bc-4ffa-b509-8ca318502ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:24.038027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867f05b-26bc-4ffa-b509-8ca318502ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.038372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867f05b-26bc-4ffa-b509-8ca318502ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.038755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867f05b-26bc-4ffa-b509-8ca318502ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.058275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867f05b-26bc-4ffa-b509-8ca318502ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:24.058275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867f05b-26bc-4ffa-b509-8ca318502ff7 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:24.076054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 907/1838] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:24.091668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 932/1839] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:23 2022] POST /identity/v3/users => generated 295 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:24.098475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b75cf-203e-46c0-8831-de34f2d82db3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.099360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149b3a6-6c4f-4436-8cab-5c3d82473dea None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.100620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149b3a6-6c4f-4436-8cab-5c3d82473dea None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.100994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b75cf-203e-46c0-8831-de34f2d82db3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.101364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149b3a6-6c4f-4436-8cab-5c3d82473dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.101738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b75cf-203e-46c0-8831-de34f2d82db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.169787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f149b3a6-6c4f-4436-8cab-5c3d82473dea None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:24.178057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f149b3a6-6c4f-4436-8cab-5c3d82473dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:24.215363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:24.217031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f56b75cf-203e-46c0-8831-de34f2d82db3 None None] MFA Rules not processed for user `eb13886f8b7040c1846bd68bd946745e`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:24.218359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f56b75cf-203e-46c0-8831-de34f2d82db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:24.229891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 908/1840] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 24 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:24.229891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149b3a6-6c4f-4436-8cab-5c3d82473dea None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:24.270110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149b3a6-6c4f-4436-8cab-5c3d82473dea None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:24.284305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:24.285150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.287610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.287994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.288674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:24.294094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:24.314826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 909/1841] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:24.316143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:24.318695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:24.337826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4242cbd-6a54-47b5-9011-f11b041ad857 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:24.349060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:24.351791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.354427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.355781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.389389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:24.391127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:24.462043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 910/1842] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:24.464579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:24.496900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f57a1-b446-47c7-90d9-e8058847a552 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:24.510077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 933/1843] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:24.515374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:24.516228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.516509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.516778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.519398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b75cf-203e-46c0-8831-de34f2d82db3 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:24.545876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:24.547368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:24.550018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b75cf-203e-46c0-8831-de34f2d82db3 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:24.569775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:24.571428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.571874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.572417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] PATH_INFO: `/v3/projects/fd9d1b918bef43378d63d0a3c5f9b1e4/users/19c5bbb335574bb0ba413d5492669f8b/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.578043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 911/1844] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:24.581755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:24.619060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ebbf4-e381-429c-800a-e34e43e3acef service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:24.630210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9d1b918bef43378d63d0a3c5f9b1e4, user_id=19c5bbb335574bb0ba413d5492669f8b, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:24.632250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:24.637359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.637359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.637359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.637359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:24.637359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:24.643081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:24.758167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 934/1845] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:24 2022] PUT /identity/v3/projects/fd9d1b918bef43378d63d0a3c5f9b1e4/users/19c5bbb335574bb0ba413d5492669f8b/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:24.760455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 912/1846] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:24 2022] POST /identity/v3/projects => generated 407 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:24.764668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:24.767912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:24.914977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04a12d-e379-45fa-810f-94d76f8c9470 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:24.961068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:24.965276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cfb85-0987-4c3a-81ff-41f9b8ee517d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:24.977985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:24.978763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:24.979488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:24.979488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:24.980342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:24.982404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.065613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:25.066552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.067607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.068460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.069458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:25.071927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.094542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 935/1847] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:25.102499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:25.123980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 913/1848] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:24 2022] POST /identity/v3/users => generated 310 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:25.128574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:25.134913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ba8b-b6df-493b-b1cf-ea07698de461 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:25.144120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce57354-674c-46e2-91b9-9f0960b863e2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:25.147265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68da502-17a7-487d-ba92-3458ca985244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:25.148180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68da502-17a7-487d-ba92-3458ca985244 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.148469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68da502-17a7-487d-ba92-3458ca985244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.148818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68da502-17a7-487d-ba92-3458ca985244 admin admin] PATH_INFO: `/v3/projects/fd9d1b918bef43378d63d0a3c5f9b1e4/users/19c5bbb335574bb0ba413d5492669f8b/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.152061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:25.155226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68da502-17a7-487d-ba92-3458ca985244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9d1b918bef43378d63d0a3c5f9b1e4, user_id=19c5bbb335574bb0ba413d5492669f8b, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:25.155362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.155362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.155362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.155362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:25.155698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68da502-17a7-487d-ba92-3458ca985244 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.157452 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.173105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 914/1849] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:25.179012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:25.179603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 936/1850] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:25 2022] PUT /identity/v3/projects/fd9d1b918bef43378d63d0a3c5f9b1e4/users/19c5bbb335574bb0ba413d5492669f8b/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:25.190310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df53da0-a94d-4685-ad67-9c9575ab7624 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:25.192294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.192869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.194238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.262087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:25.271511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] MFA Rules not processed for user `19c5bbb335574bb0ba413d5492669f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:25.275042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:25.298314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:25.301022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.301636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.302066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] PATH_INFO: `/v3/projects/c45f3c23c3234b078b72f10c08a78aec/users/c2e804afde26458f882a331abff68cb8/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.334419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45f3c23c3234b078b72f10c08a78aec, user_id=c2e804afde26458f882a331abff68cb8, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:25.337112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.417741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 937/1851] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:25.436510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:25.438430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:25.487177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 915/1852] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:25 2022] PUT /identity/v3/projects/c45f3c23c3234b078b72f10c08a78aec/users/c2e804afde26458f882a331abff68cb8/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:30:25.491245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:25.614176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7857a-3d69-4f77-b16f-e9b1a2613a12 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:25.662870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b19fbb-13c6-4ade-a81c-a893982eeddf admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:25.668394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:25.669236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.669731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.670377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.671302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:25.673118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.684831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 916/1853] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:25.688999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:25.716845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:25.719434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.719434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.719434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.729458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b17972-2428-44d0-adbf-9f1bea11ac80 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:25.735633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276585ec-8a3c-42fb-a64b-3e46db52c39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:25.736675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276585ec-8a3c-42fb-a64b-3e46db52c39f admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.737108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276585ec-8a3c-42fb-a64b-3e46db52c39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.737421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276585ec-8a3c-42fb-a64b-3e46db52c39f admin admin] PATH_INFO: `/v3/projects/c45f3c23c3234b078b72f10c08a78aec/users/c2e804afde26458f882a331abff68cb8/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.740319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276585ec-8a3c-42fb-a64b-3e46db52c39f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45f3c23c3234b078b72f10c08a78aec, user_id=c2e804afde26458f882a331abff68cb8, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:25.745239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276585ec-8a3c-42fb-a64b-3e46db52c39f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.770734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 917/1854] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:25 2022] PUT /identity/v3/projects/c45f3c23c3234b078b72f10c08a78aec/users/c2e804afde26458f882a331abff68cb8/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:25.772651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60fa22-52e5-4807-9954-bda09c368174 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:25.773231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60fa22-52e5-4807-9954-bda09c368174 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:25.774099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae60fa22-52e5-4807-9954-bda09c368174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:25.836894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ae60fa22-52e5-4807-9954-bda09c368174 None None] MFA Rules not processed for user `c2e804afde26458f882a331abff68cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:25.837417 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:25.839738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ae60fa22-52e5-4807-9954-bda09c368174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:25.845778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:25.845778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:25.886302 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 938/1855] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:25.978836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 918/1856] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:25.994601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:25.997047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:26.024119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fea78b-40c0-43df-8466-255e449e8fee service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:26.039332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:26.115468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:26.116216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:26.116588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:26.116904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:26.141361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:26.152329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:26.176826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 939/1857] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:27.080971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108bdcc-4cfa-4924-921b-4c18242bb273 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.081480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108bdcc-4cfa-4924-921b-4c18242bb273 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.081574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108bdcc-4cfa-4924-921b-4c18242bb273 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.083924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 919/1858] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:27.088431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:27.094891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:27.102973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f4fd9-f67e-4b4a-a480-52ee377d4ed3 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.103564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f4fd9-f67e-4b4a-a480-52ee377d4ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.104350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f4fd9-f67e-4b4a-a480-52ee377d4ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.185040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-287f4fd9-f67e-4b4a-a480-52ee377d4ed3 None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:27.185040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-287f4fd9-f67e-4b4a-a480-52ee377d4ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:27.186610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:27.216625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171748d-12b0-4609-b518-90e5c95a6e46 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:27.229208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:27.229208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.229208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.229208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] PATH_INFO: `/v3/projects/0871f3f2a71d43d3b96c3a9c797a358c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.230508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0871f3f2a71d43d3b96c3a9c797a358c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:27.232529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:27.280794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:27.313215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 920/1859] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:27.314156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f4fd9-f67e-4b4a-a480-52ee377d4ed3 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:27.330792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f4fd9-f67e-4b4a-a480-52ee377d4ed3 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:27.334501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:27.335128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.335407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.335685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.412236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0871f3f2a71d43d3b96c3a9c797a358c', 'request_id': 'req-162b04c8-9d40-4742-805f-f2fce2bb9c05'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:27.420435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0871f3f2a71d43d3b96c3a9c797a358c', 'request_id': 'req-162b04c8-9d40-4742-805f-f2fce2bb9c05'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:27.428296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0871f3f2a71d43d3b96c3a9c797a358c', 'request_id': 'req-162b04c8-9d40-4742-805f-f2fce2bb9c05'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:27.434384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:27.436421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:27.447113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 921/1860] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:27 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:27.448021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:27.449488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:27.466550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98531d6-1c60-4866-ab95-8cc721563659 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:27.545604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae8cba-ff5e-45ae-b3dd-d18345a74e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:27.547166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8cba-ff5e-45ae-b3dd-d18345a74e04 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.548007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8cba-ff5e-45ae-b3dd-d18345a74e04 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.548658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae8cba-ff5e-45ae-b3dd-d18345a74e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.555272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] The token cache is being invalidate because project 0871f3f2a71d43d3b96c3a9c797a358c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:27.555560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:27.557744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:27.560501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 940/1861] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:27 2022] DELETE /identity/v3/projects/0871f3f2a71d43d3b96c3a9c797a358c => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:27.564344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae8cba-ff5e-45ae-b3dd-d18345a74e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:27.565338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:27.566366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae8cba-ff5e-45ae-b3dd-d18345a74e04 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:27.567232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:27.581969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 922/1862] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:27.649003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ce99-3e2b-4dc3-aac4-282279f55b07 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.649380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ce99-3e2b-4dc3-aac4-282279f55b07 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.649659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ce99-3e2b-4dc3-aac4-282279f55b07 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.651412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 923/1863] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:27.671597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.672054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.672494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.697466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b04c8-9d40-4742-805f-f2fce2bb9c05 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:27.701504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:27.703039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:27.703631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:27.704893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.705170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.705444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] PATH_INFO: `/v3/users/cddd95c4e6984e82baa7c954e13dec76` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.706813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cddd95c4e6984e82baa7c954e13dec76)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:27.709104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:27.712854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 924/1864] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:27.730854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:27.735998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:27.751550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb35e2-cba6-4e92-b3cf-5bc2dbe3d44a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:27.757590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:27.758184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:27.758462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:27.758762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:27.767285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:27.768840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:27.779951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 925/1865] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:28.015392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cddd95c4e6984e82baa7c954e13dec76'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.016695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cddd95c4e6984e82baa7c954e13dec76'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.102681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cddd95c4e6984e82baa7c954e13dec76'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.115987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cddd95c4e6984e82baa7c954e13dec76'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.116699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8ebe8a38-7d9d-4686-ad82-49c943c3c1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cddd95c4e6984e82baa7c954e13dec76'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.131036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 941/1866] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:27 2022] DELETE /identity/v3/users/cddd95c4e6984e82baa7c954e13dec76 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:28.609635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:28.620511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc534d-1eae-457a-a93f-c887ddab126e service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:28.678214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:28.679057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:28.679336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:28.679609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] PATH_INFO: `/v3/projects/51eb54a16bc74d1c9ec55bb093932920` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:28.680805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51eb54a16bc74d1c9ec55bb093932920)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:28.682871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:28.791896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51eb54a16bc74d1c9ec55bb093932920', 'request_id': 'req-aebc2051-677e-4b56-bea4-3228fc184f34'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.798856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51eb54a16bc74d1c9ec55bb093932920', 'request_id': 'req-aebc2051-677e-4b56-bea4-3228fc184f34'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.804993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51eb54a16bc74d1c9ec55bb093932920', 'request_id': 'req-aebc2051-677e-4b56-bea4-3228fc184f34'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.904633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] The token cache is being invalidate because project 51eb54a16bc74d1c9ec55bb093932920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:28.904633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.904633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-aebc2051-677e-4b56-bea4-3228fc184f34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:28.905109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 926/1867] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:28 2022] DELETE /identity/v3/projects/51eb54a16bc74d1c9ec55bb093932920 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:28.910982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321ae8-52a3-4601-9a25-a8efbb71c9be None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:28.911263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321ae8-52a3-4601-9a25-a8efbb71c9be None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:28.911688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321ae8-52a3-4601-9a25-a8efbb71c9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:28.932168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ef321ae8-52a3-4601-9a25-a8efbb71c9be None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:28.933416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ef321ae8-52a3-4601-9a25-a8efbb71c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:28.996937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 942/1868] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:29.005308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a0025-dd60-4041-9376-5b1d8034262d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.005678 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a0025-dd60-4041-9376-5b1d8034262d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.006023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a0025-dd60-4041-9376-5b1d8034262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.027019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7d4a0025-dd60-4041-9376-5b1d8034262d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:29.030313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7d4a0025-dd60-4041-9376-5b1d8034262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:29.036195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 927/1869] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:29.040474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321ae8-52a3-4601-9a25-a8efbb71c9be None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.050665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321ae8-52a3-4601-9a25-a8efbb71c9be None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.059345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.059345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.059345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.059345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.060068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:29.060068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.060068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.073937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 943/1870] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:29.079272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a0025-dd60-4041-9376-5b1d8034262d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.091920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a0025-dd60-4041-9376-5b1d8034262d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.103878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.104528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.104807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.105072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.105739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.108090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.146449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:29.204332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 928/1871] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:29 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:29.209249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.227343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacf4c2-acda-4047-87c5-a32c46cbf18e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.295943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.296556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.296838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.297477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.298171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.300375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.380148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 944/1872] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:29 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:29.382012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.392079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a0907-878a-4fdd-ac54-923a86524202 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.396728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.397453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.397627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.397898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.398540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.400141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.411143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 929/1873] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:29.415198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.425437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47194e34-bbf5-41f5-94a0-d2fa1e199b9a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.435707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81578a81-bb95-4081-a05f-f4114f950e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.435707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578a81-bb95-4081-a05f-f4114f950e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.435707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578a81-bb95-4081-a05f-f4114f950e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.435707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578a81-bb95-4081-a05f-f4114f950e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.436053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81578a81-bb95-4081-a05f-f4114f950e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.436053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81578a81-bb95-4081-a05f-f4114f950e34 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.446050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 945/1874] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:29.450332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.460128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e485b-06d3-4b56-8c4f-9226713470dc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.464730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.465360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.465635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.465903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] PATH_INFO: `/v3/projects/df2efa8dff954995952ce1b0d7b5f3c6/users/5e29bfbf5e1f4f30b72189e722213589/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.480176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2efa8dff954995952ce1b0d7b5f3c6, user_id=5e29bfbf5e1f4f30b72189e722213589, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.482920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.551527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 930/1875] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:29 2022] PUT /identity/v3/projects/df2efa8dff954995952ce1b0d7b5f3c6/users/5e29bfbf5e1f4f30b72189e722213589/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:29.561665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a438-b11a-4ebc-9c51-97ac98c82e5d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.561951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a438-b11a-4ebc-9c51-97ac98c82e5d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.562222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a438-b11a-4ebc-9c51-97ac98c82e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.590009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6945a438-b11a-4ebc-9c51-97ac98c82e5d None None] MFA Rules not processed for user `5e29bfbf5e1f4f30b72189e722213589`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:29.591257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6945a438-b11a-4ebc-9c51-97ac98c82e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:29.653902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 946/1876] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:29.658610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.662409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:29.722138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d7f55-3c41-4ff7-881d-9017c41d920d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.727602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.728206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.728466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.728771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.729402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.731358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.799650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 931/1877] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:29 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:29.804282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a438-b11a-4ebc-9c51-97ac98c82e5d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.814809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a438-b11a-4ebc-9c51-97ac98c82e5d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.875703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.876303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.876576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.876842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.877496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.879427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.967605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 947/1878] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:29 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:29.969142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:29.980269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bc3f1-5a70-40b0-81a9-8e55c1442a57 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:29.984969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:29.985562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.985875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.986148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.986816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:29.989030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:29.999749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 932/1879] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:30.003983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:30.013793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6998eb-d9b8-4d35-b8bb-b6cf0865a907 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:30.018323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:30.022800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.022800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.022800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] PATH_INFO: `/v3/projects/ec6ebc8b9afe4ace9e72482157d32cf8/users/25bf61f91cab4eb9b865d3bdaa4f04a3/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.033701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6ebc8b9afe4ace9e72482157d32cf8, user_id=25bf61f91cab4eb9b865d3bdaa4f04a3, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:30.036901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:30.111103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 948/1880] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:30 2022] PUT /identity/v3/projects/ec6ebc8b9afe4ace9e72482157d32cf8/users/25bf61f91cab4eb9b865d3bdaa4f04a3/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:30.115402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:30.125098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b4c01-aa39-4cbc-84c0-b1d0cdef6bae admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:30.181433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbc17a-e936-4fa4-a8f7-26d4d04aa2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:30.182033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbc17a-e936-4fa4-a8f7-26d4d04aa2cf admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.182310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbc17a-e936-4fa4-a8f7-26d4d04aa2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.182596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbc17a-e936-4fa4-a8f7-26d4d04aa2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.183406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbc17a-e936-4fa4-a8f7-26d4d04aa2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:30.184988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbc17a-e936-4fa4-a8f7-26d4d04aa2cf admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:30.196078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 933/1881] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:30.199850 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:30.210685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e2581-59ae-4e41-9616-879934666689 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:30.215248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:30.215890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.216168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.216440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] PATH_INFO: `/v3/projects/ec6ebc8b9afe4ace9e72482157d32cf8/users/25bf61f91cab4eb9b865d3bdaa4f04a3/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.218378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6ebc8b9afe4ace9e72482157d32cf8, user_id=25bf61f91cab4eb9b865d3bdaa4f04a3, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:30.221105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:30.236878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 949/1882] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:30 2022] PUT /identity/v3/projects/ec6ebc8b9afe4ace9e72482157d32cf8/users/25bf61f91cab4eb9b865d3bdaa4f04a3/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:30.247966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4524-f731-4074-8c94-ba7d8cbd3962 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.248255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4524-f731-4074-8c94-ba7d8cbd3962 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.248556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4524-f731-4074-8c94-ba7d8cbd3962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.258178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:30.302905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-841f4524-f731-4074-8c94-ba7d8cbd3962 None None] MFA Rules not processed for user `25bf61f91cab4eb9b865d3bdaa4f04a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:30.304114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-841f4524-f731-4074-8c94-ba7d8cbd3962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:30.377391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 934/1883] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:30.389634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:30.391849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:30.468062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836d95-fd2c-4d63-a7cd-f5d0f2815e73 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:30.480644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae3edd-8a91-4c56-af85-16d49fdd1aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:30.480644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae3edd-8a91-4c56-af85-16d49fdd1aad service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.480644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae3edd-8a91-4c56-af85-16d49fdd1aad service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.480644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae3edd-8a91-4c56-af85-16d49fdd1aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.489201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae3edd-8a91-4c56-af85-16d49fdd1aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:30.490976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae3edd-8a91-4c56-af85-16d49fdd1aad service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:30.501299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 950/1884] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:30.799085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:30.828120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:31.057408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:31.217628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4524-f731-4074-8c94-ba7d8cbd3962 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:31.310248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4524-f731-4074-8c94-ba7d8cbd3962 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:31.323795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:31.324444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:31.324895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:31.325017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] PATH_INFO: `/v3/users/5e29bfbf5e1f4f30b72189e722213589` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:31.326486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e29bfbf5e1f4f30b72189e722213589)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:31.328384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:31.544154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:31.547307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e29bfbf5e1f4f30b72189e722213589'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:31.547474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e29bfbf5e1f4f30b72189e722213589'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:31.551399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e29bfbf5e1f4f30b72189e722213589'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:31.600926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:31.603666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e29bfbf5e1f4f30b72189e722213589'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:31.634960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e29bfbf5e1f4f30b72189e722213589'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:31.742329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 935/1885] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:31 2022] DELETE /identity/v3/users/5e29bfbf5e1f4f30b72189e722213589 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:31.742429 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:31.742621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:31.743006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:31.790225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:31.791750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:31.883648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 951/1886] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:31.896497 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:31.898034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:31.959318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a7b31-572c-478f-b210-d686c6e3aad2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:31.963396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:31.964003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:31.964306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:31.964575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:31.973898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:31.975422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:31.985070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 936/1887] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:32.033603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:32.191970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:32.567194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:32.571035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:32.596428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7b7e7-035c-4955-bf19-797a582cfffc None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:32.609341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:32.610136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:32.610492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:32.610917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] PATH_INFO: `/v3/projects/df2efa8dff954995952ce1b0d7b5f3c6` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:32.616021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2efa8dff954995952ce1b0d7b5f3c6)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:32.619218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:32.803024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:32.924229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2efa8dff954995952ce1b0d7b5f3c6', 'request_id': 'req-ad1355be-d3f5-416b-8b18-38f49415bf60'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:32.947278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2efa8dff954995952ce1b0d7b5f3c6', 'request_id': 'req-ad1355be-d3f5-416b-8b18-38f49415bf60'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:32.955439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2efa8dff954995952ce1b0d7b5f3c6', 'request_id': 'req-ad1355be-d3f5-416b-8b18-38f49415bf60'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.059797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] The token cache is being invalidate because project df2efa8dff954995952ce1b0d7b5f3c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:33.078396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.078396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.078396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 952/1888] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:32 2022] DELETE /identity/v3/projects/df2efa8dff954995952ce1b0d7b5f3c6 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:33.081749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:33.101138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f414af-c6c3-4db3-b5e7-a9ee013793a7 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:33.103488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:33.174261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:33.175123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:33.176078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:33.176423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] PATH_INFO: `/v3/users/25bf61f91cab4eb9b865d3bdaa4f04a3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:33.187242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] RBAC: Authorizing `identity:delete_user(user_id=25bf61f91cab4eb9b865d3bdaa4f04a3)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:33.189715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:33.198059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:33.344337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25bf61f91cab4eb9b865d3bdaa4f04a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.345113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25bf61f91cab4eb9b865d3bdaa4f04a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.345722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25bf61f91cab4eb9b865d3bdaa4f04a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.354047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25bf61f91cab4eb9b865d3bdaa4f04a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.369026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-302775ef-0155-49f1-8247-47ac4f37dfca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25bf61f91cab4eb9b865d3bdaa4f04a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:33.482318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 937/1889] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:33 2022] DELETE /identity/v3/users/25bf61f91cab4eb9b865d3bdaa4f04a3 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:33.565707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:33.606944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:33.789574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:33.839285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:33.971909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:34.180131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:34.516124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:34.517903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:34.546639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1355be-d3f5-416b-8b18-38f49415bf60 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:34.709466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:34.710401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:34.710896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:34.711490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] PATH_INFO: `/v3/projects/ec6ebc8b9afe4ace9e72482157d32cf8` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:34.713826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6ebc8b9afe4ace9e72482157d32cf8)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:34.717263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:34.821796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:35.060923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6ebc8b9afe4ace9e72482157d32cf8', 'request_id': 'req-151c578d-4aca-4dc8-98a8-81057ed40834'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:35.070045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6ebc8b9afe4ace9e72482157d32cf8', 'request_id': 'req-151c578d-4aca-4dc8-98a8-81057ed40834'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:35.091459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6ebc8b9afe4ace9e72482157d32cf8', 'request_id': 'req-151c578d-4aca-4dc8-98a8-81057ed40834'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:35.841398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] The token cache is being invalidate because project ec6ebc8b9afe4ace9e72482157d32cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:35.841398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:35.841398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-151c578d-4aca-4dc8-98a8-81057ed40834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:35.841398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 953/1890] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:34 2022] DELETE /identity/v3/projects/ec6ebc8b9afe4ace9e72482157d32cf8 => generated 0 bytes in 1315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:35.845108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537478ac-ea2e-482c-9f29-e1cf5c60171f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:35.845548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537478ac-ea2e-482c-9f29-e1cf5c60171f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:35.845643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537478ac-ea2e-482c-9f29-e1cf5c60171f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:35.874063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-537478ac-ea2e-482c-9f29-e1cf5c60171f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:35.878795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-537478ac-ea2e-482c-9f29-e1cf5c60171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:35.960805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 938/1891] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:35.964601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf364b7a-0189-4059-8c14-278987f6d054 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:35.965275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf364b7a-0189-4059-8c14-278987f6d054 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:35.965893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf364b7a-0189-4059-8c14-278987f6d054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:35.998440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bf364b7a-0189-4059-8c14-278987f6d054 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:36.004534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bf364b7a-0189-4059-8c14-278987f6d054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:36.018464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 954/1892] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:36.019839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537478ac-ea2e-482c-9f29-e1cf5c60171f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:36.035428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537478ac-ea2e-482c-9f29-e1cf5c60171f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:36.041792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:36.044654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:36.044654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:36.044654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:36.044654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:36.044654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:36.045697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:36.088941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 939/1893] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:36.090348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf364b7a-0189-4059-8c14-278987f6d054 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:36.106566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf364b7a-0189-4059-8c14-278987f6d054 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:36.112735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:36.117443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:36.117443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:36.117443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:36.117443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:36.119748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:36.258178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 955/1894] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:36 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:36.263034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:36.293389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41906b63-4119-4cf9-aae2-53c92e5c8b01 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:36.374782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:36.399517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:36.400140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:36.400422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:36.400775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:36.401448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:36.405384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:36.534688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 940/1895] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:36 2022] POST /identity/v3/users => generated 291 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:36.537736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:36.553860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c53fc-24c0-4a84-9f46-8e1e46d5a3f8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:36.562013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:36.581339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:36.581339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:36.581339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:36.581339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:36.581339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:36.581339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 956/1896] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:36.583293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:36.593425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e205f3-6773-4458-9172-0783706f1032 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:36.598686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072ef92-a815-497c-8150-88e78fea375a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:36.599285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ef92-a815-497c-8150-88e78fea375a admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:36.599564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ef92-a815-497c-8150-88e78fea375a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:36.599953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072ef92-a815-497c-8150-88e78fea375a admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:36.600598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072ef92-a815-497c-8150-88e78fea375a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:36.602198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072ef92-a815-497c-8150-88e78fea375a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:36.617187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 941/1897] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:36.621325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:36.631930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07816c37-f013-494f-9f0b-4810c38e0870 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:36.636547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:36.637198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:36.637478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:36.637751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] PATH_INFO: `/v3/projects/7a3719c7810d49d3abfff5befe6eceb9/users/31aee5e3eb134729b1448cfa60da6de0/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:36.655473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3719c7810d49d3abfff5befe6eceb9, user_id=31aee5e3eb134729b1448cfa60da6de0, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:36.664229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:36.754208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 957/1898] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:36 2022] PUT /identity/v3/projects/7a3719c7810d49d3abfff5befe6eceb9/users/31aee5e3eb134729b1448cfa60da6de0/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:30:36.766584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab7a2b-e69c-4054-a538-34f18b6346e8 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:36.766899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab7a2b-e69c-4054-a538-34f18b6346e8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:36.768038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab7a2b-e69c-4054-a538-34f18b6346e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:36.816543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c2ab7a2b-e69c-4054-a538-34f18b6346e8 None None] MFA Rules not processed for user `31aee5e3eb134729b1448cfa60da6de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:36.818072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c2ab7a2b-e69c-4054-a538-34f18b6346e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:36.908233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 942/1899] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:36.913587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:36.920838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:36.991176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:37.011941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e2b72-f5d7-445e-8f3e-9ec355b18304 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:37.025203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:37.025878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:37.026107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:37.026376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:37.027080 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:37.029065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:37.357514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 958/1900] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:36 2022] POST /identity/v3/projects => generated 383 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:37.363987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab7a2b-e69c-4054-a538-34f18b6346e8 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:37.382320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab7a2b-e69c-4054-a538-34f18b6346e8 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:37.491492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:37.492110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:37.492486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:37.492776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:37.493390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:37.496559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:37.676981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 943/1901] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:37 2022] POST /identity/v3/users => generated 298 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:37.682790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:37.719100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df16b0-6073-4460-b9b4-e952875eb91c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:37.731358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:37.733763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:37.735135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:37.737447 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:37.738876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:37.743668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:37.768042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 959/1902] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:37.768300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:37.777366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3266ece-1e30-4235-b11e-5e0a36a1aef1 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:37.793825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:37.794680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:37.795062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:37.795436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] PATH_INFO: `/v3/projects/2128bb7f6f98478fa5e70c4003ecf103/users/8cbb5056f0cc4fefb2527f24d973f903/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:37.822582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2128bb7f6f98478fa5e70c4003ecf103, user_id=8cbb5056f0cc4fefb2527f24d973f903, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:37.825537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:37.932711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 944/1903] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:37 2022] PUT /identity/v3/projects/2128bb7f6f98478fa5e70c4003ecf103/users/8cbb5056f0cc4fefb2527f24d973f903/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:37.949343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:37.969481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438114c9-a891-4d67-9032-36e3cdf66728 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:38.073604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c03136-9a3b-4eab-9d5d-1c4455c61bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:38.074381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03136-9a3b-4eab-9d5d-1c4455c61bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:38.074759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03136-9a3b-4eab-9d5d-1c4455c61bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:38.075302 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03136-9a3b-4eab-9d5d-1c4455c61bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:38.076173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c03136-9a3b-4eab-9d5d-1c4455c61bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:38.078031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c03136-9a3b-4eab-9d5d-1c4455c61bd3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:38.093202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 960/1904] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:38.098195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:38.131981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf6c27-450b-4045-b257-ae3b9e283e43 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:38.148222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:38.149214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:38.149600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:38.150233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] PATH_INFO: `/v3/projects/2128bb7f6f98478fa5e70c4003ecf103/users/8cbb5056f0cc4fefb2527f24d973f903/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:38.155790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2128bb7f6f98478fa5e70c4003ecf103, user_id=8cbb5056f0cc4fefb2527f24d973f903, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:38.159506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:38.188317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 945/1905] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:38 2022] PUT /identity/v3/projects/2128bb7f6f98478fa5e70c4003ecf103/users/8cbb5056f0cc4fefb2527f24d973f903/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:38.199654 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc27c4-0afa-45f1-8d7d-ad6787141548 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:38.200580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc27c4-0afa-45f1-8d7d-ad6787141548 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:38.200580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc27c4-0afa-45f1-8d7d-ad6787141548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:38.242271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-eebc27c4-0afa-45f1-8d7d-ad6787141548 None None] MFA Rules not processed for user `8cbb5056f0cc4fefb2527f24d973f903`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:38.243533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-eebc27c4-0afa-45f1-8d7d-ad6787141548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:38.333463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:38.341786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 961/1906] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:38.358878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:38.363243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:38.421752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:38.455670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2179b8-d6c0-46c4-afd2-fd55183d1418 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:38.460063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:38.460775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:38.461118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:38.461457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:38.471143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:38.473442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:38.483425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 946/1907] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:38.666258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc27c4-0afa-45f1-8d7d-ad6787141548 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:38.677389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc27c4-0afa-45f1-8d7d-ad6787141548 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:38.681301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068457f-296b-4db9-8d6f-8343740b90ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:38.682429 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068457f-296b-4db9-8d6f-8343740b90ed service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:38.683123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068457f-296b-4db9-8d6f-8343740b90ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:38.683670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068457f-296b-4db9-8d6f-8343740b90ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:38.760316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068457f-296b-4db9-8d6f-8343740b90ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:38.762194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068457f-296b-4db9-8d6f-8343740b90ed service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:38.774429 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 962/1908] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:39.443002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:39.531671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7edc4f-1d9f-43a2-8235-ff20fd76e82e service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:39.537874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:39.538564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:39.538877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:39.539150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] PATH_INFO: `/v3/users/31aee5e3eb134729b1448cfa60da6de0` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:39.540386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31aee5e3eb134729b1448cfa60da6de0)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:39.542302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:39.725726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31aee5e3eb134729b1448cfa60da6de0'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:39.726374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31aee5e3eb134729b1448cfa60da6de0'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:39.727010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31aee5e3eb134729b1448cfa60da6de0'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:39.734243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31aee5e3eb134729b1448cfa60da6de0'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:39.747435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31aee5e3eb134729b1448cfa60da6de0'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:39.859701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 947/1909] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:39 2022] DELETE /identity/v3/users/31aee5e3eb134729b1448cfa60da6de0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:39.865179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:39.865674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:39.865784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:39.889424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:39.890696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:39.965355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 963/1910] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:39.975115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:39.976600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:40.031209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:40.035335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c7ea-34c0-4bc8-b782-48b6695b1940 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:40.039129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:40.039749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:40.040025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:40.040294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:40.049614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:40.051591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:40.061283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 948/1911] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:40.877535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:40.880070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:40.895030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e061378-9e8b-4b31-9fe0-8dcc4c782a74 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:40.901734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:40.902933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:40.903398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:40.903673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] PATH_INFO: `/v3/projects/7a3719c7810d49d3abfff5befe6eceb9` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:40.905393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3719c7810d49d3abfff5befe6eceb9)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:40.907760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:41.043231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3719c7810d49d3abfff5befe6eceb9', 'request_id': 'req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.051096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3719c7810d49d3abfff5befe6eceb9', 'request_id': 'req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.058497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3719c7810d49d3abfff5befe6eceb9', 'request_id': 'req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.144893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] The token cache is being invalidate because project 7a3719c7810d49d3abfff5befe6eceb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:41.145176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.145807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.147444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 964/1912] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:40 2022] DELETE /identity/v3/projects/7a3719c7810d49d3abfff5befe6eceb9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:41.151509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:41.165197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5ad61-55d6-401c-af51-b5d5912467dd service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:41.262575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:41.264627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:41.265088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:41.265533 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] PATH_INFO: `/v3/users/8cbb5056f0cc4fefb2527f24d973f903` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:41.267067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbb5056f0cc4fefb2527f24d973f903)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:41.269914 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:41.452295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:41.458637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:41.466073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbb5056f0cc4fefb2527f24d973f903'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.467387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbb5056f0cc4fefb2527f24d973f903'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.468239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbb5056f0cc4fefb2527f24d973f903'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.481149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbb5056f0cc4fefb2527f24d973f903'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.511236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbb5056f0cc4fefb2527f24d973f903'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:41.516409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:41.589348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c201e6-4e9b-4df9-92bc-9e1c00c62ac8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:41.593628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d895c-5a47-443d-8f5f-d9c940f791f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:41.594415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d895c-5a47-443d-8f5f-d9c940f791f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:41.595443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d895c-5a47-443d-8f5f-d9c940f791f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:41.596065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d895c-5a47-443d-8f5f-d9c940f791f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:41.598514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-321d895c-5a47-443d-8f5f-d9c940f791f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:41.608819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 949/1913] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:41 2022] DELETE /identity/v3/users/8cbb5056f0cc4fefb2527f24d973f903 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:41.701462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d895c-5a47-443d-8f5f-d9c940f791f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:41.707025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d895c-5a47-443d-8f5f-d9c940f791f2 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:41.719866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 965/1914] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:42.274589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:42.276171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:42.349000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e705c4-beb7-40ef-a38c-56041d80cf8a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:42.356447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:42.356447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.356447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.356447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] PATH_INFO: `/v3/projects/2128bb7f6f98478fa5e70c4003ecf103` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.356809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2128bb7f6f98478fa5e70c4003ecf103)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:42.358327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:42.375044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:42.479815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2128bb7f6f98478fa5e70c4003ecf103', 'request_id': 'req-9d2720ce-175b-46f1-bd4c-16d3b511192b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:42.486012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2128bb7f6f98478fa5e70c4003ecf103', 'request_id': 'req-9d2720ce-175b-46f1-bd4c-16d3b511192b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:42.492587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2128bb7f6f98478fa5e70c4003ecf103', 'request_id': 'req-9d2720ce-175b-46f1-bd4c-16d3b511192b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:42.602334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] The token cache is being invalidate because project 2128bb7f6f98478fa5e70c4003ecf103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:42.607000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:42.607000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9d2720ce-175b-46f1-bd4c-16d3b511192b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:42.607000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 950/1915] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:42 2022] DELETE /identity/v3/projects/2128bb7f6f98478fa5e70c4003ecf103 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:42.610942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a40e0b-1038-40b8-9ddf-4ebc662d51ad None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.611398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a40e0b-1038-40b8-9ddf-4ebc662d51ad None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.611669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a40e0b-1038-40b8-9ddf-4ebc662d51ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.641402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-28a40e0b-1038-40b8-9ddf-4ebc662d51ad None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:42.645265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-28a40e0b-1038-40b8-9ddf-4ebc662d51ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:42.709048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 966/1916] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:42.717385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24624b33-bcad-47b6-9e65-d9ad61cd427b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.717670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24624b33-bcad-47b6-9e65-d9ad61cd427b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.717941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24624b33-bcad-47b6-9e65-d9ad61cd427b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.740998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-24624b33-bcad-47b6-9e65-d9ad61cd427b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:42.742422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-24624b33-bcad-47b6-9e65-d9ad61cd427b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:42.751092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 951/1917] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:42.756194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a40e0b-1038-40b8-9ddf-4ebc662d51ad None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:42.765615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a40e0b-1038-40b8-9ddf-4ebc662d51ad None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:42.770183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:42.770815 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.771091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.771358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.771994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:42.772286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:42.774041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:42.791402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 967/1918] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:30:42.794328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24624b33-bcad-47b6-9e65-d9ad61cd427b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:42.808255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24624b33-bcad-47b6-9e65-d9ad61cd427b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:42.813483 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:42.814006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.814281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.814671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.815305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:42.817370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:42.916729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 952/1919] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:42 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:42.917846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:42.929066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73f98c-1983-479c-ae39-f079301494b6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:43.011114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:43.011723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.012002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.012272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.012927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:43.014995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:43.025130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:43.108405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 968/1920] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:42 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:43.112857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:43.126259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c1f7a-f294-45ab-a80e-1a4cca2d8123 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:43.135532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:43.136195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.136474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.137387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.138049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:43.139961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:43.154602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 953/1921] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:43.155979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:43.169985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b483345-9fe7-4ee2-b396-fde84c15c2cf admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:43.176553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708bca0-8439-47e8-b48b-a586070ad9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:43.177202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708bca0-8439-47e8-b48b-a586070ad9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.177484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708bca0-8439-47e8-b48b-a586070ad9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.177757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708bca0-8439-47e8-b48b-a586070ad9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.178404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708bca0-8439-47e8-b48b-a586070ad9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:43.180414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708bca0-8439-47e8-b48b-a586070ad9b0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:43.214946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 969/1922] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:43.218500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:43.237132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e876b-5e67-49df-ad5e-d007241d33a7 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:43.244236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:43.245187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.246175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.246839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] PATH_INFO: `/v3/projects/52c77ee5482845aaad46d0b6ecedd525/users/995a0d3db89147b1b9749338acb2f5b5/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.273788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c77ee5482845aaad46d0b6ecedd525, user_id=995a0d3db89147b1b9749338acb2f5b5, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:43.276477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:43.342988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 954/1923] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:43 2022] PUT /identity/v3/projects/52c77ee5482845aaad46d0b6ecedd525/users/995a0d3db89147b1b9749338acb2f5b5/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:43.354738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.355026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.355301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.391906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] MFA Rules not processed for user `995a0d3db89147b1b9749338acb2f5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:43.395285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:43.458105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 970/1924] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:43.468929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:43.470575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:43.549855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99777e50-1f44-4002-acb8-0f3de9d3d2e8 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:43.553499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e52c34-4f0c-456d-a6fa-d47892f7069f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f33e1c573c044f086f99eb128c8f726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:43.554091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52c34-4f0c-456d-a6fa-d47892f7069f service glance] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.554365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52c34-4f0c-456d-a6fa-d47892f7069f service glance] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.554673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52c34-4f0c-456d-a6fa-d47892f7069f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.562893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e52c34-4f0c-456d-a6fa-d47892f7069f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:43.564315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e52c34-4f0c-456d-a6fa-d47892f7069f service glance] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:43.577896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 955/1925] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:53.455659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:53.569587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:53.635656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:53.635656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:53.704724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba894223-6bd6-4c59-9361-1f5600306f84 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:53.709908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:53.710541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:53.710848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:53.711117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] PATH_INFO: `/v3/users/eb13886f8b7040c1846bd68bd946745e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:53.712227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb13886f8b7040c1846bd68bd946745e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:53.714139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:53.929954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb13886f8b7040c1846bd68bd946745e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:53.931405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb13886f8b7040c1846bd68bd946745e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.026679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb13886f8b7040c1846bd68bd946745e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.043225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb13886f8b7040c1846bd68bd946745e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.043225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb13886f8b7040c1846bd68bd946745e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.054393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 971/1926] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:53 2022] DELETE /identity/v3/users/eb13886f8b7040c1846bd68bd946745e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:54.059126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:54.059408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:54.059730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:54.078894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:54.080076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:54.136323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 956/1927] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:54.145741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:54.147165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:54.202625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34c9c-6dd2-4d27-b6f4-b232cd8e67d6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:54.206170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:54.206827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:54.207108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:54.207378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:54.216027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:54.217438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:54.226661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 972/1928] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:54.695794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:54.697489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:54.710215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05768b47-346c-4a5c-9c02-1c38059b3585 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:54.718786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:54.720061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:54.720061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:54.720061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] PATH_INFO: `/v3/projects/acf75b4993bb4dbd9dfa3630691fde9f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:54.723570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf75b4993bb4dbd9dfa3630691fde9f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:54.725599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:54.854484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf75b4993bb4dbd9dfa3630691fde9f', 'request_id': 'req-f66049f1-1d93-4f12-abd9-a972ee3e1642'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.861660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf75b4993bb4dbd9dfa3630691fde9f', 'request_id': 'req-f66049f1-1d93-4f12-abd9-a972ee3e1642'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.868853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf75b4993bb4dbd9dfa3630691fde9f', 'request_id': 'req-f66049f1-1d93-4f12-abd9-a972ee3e1642'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.947800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] The token cache is being invalidate because project acf75b4993bb4dbd9dfa3630691fde9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:54.948099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.949024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:54.950828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 957/1929] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:54 2022] DELETE /identity/v3/projects/acf75b4993bb4dbd9dfa3630691fde9f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:54.957268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:54.978836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d57b2-e10a-42db-bddb-81767edbbcbd service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:54.986594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:55.029316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:55.030030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:55.030305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:55.030571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] PATH_INFO: `/v3/users/c2e804afde26458f882a331abff68cb8` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:55.031991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e804afde26458f882a331abff68cb8)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:55.034314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:55.237176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e804afde26458f882a331abff68cb8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:55.237835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e804afde26458f882a331abff68cb8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:55.298455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e804afde26458f882a331abff68cb8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:55.310705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e804afde26458f882a331abff68cb8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:55.311352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-5d7a0e92-4e16-40bb-bf32-9ebcc9202f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e804afde26458f882a331abff68cb8'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:55.323843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 973/1930] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:30:54 2022] DELETE /identity/v3/users/c2e804afde26458f882a331abff68cb8 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:55.766916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:55.768420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:55.781652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66049f1-1d93-4f12-abd9-a972ee3e1642 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:55.846998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:55.847991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:55.847991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:55.848254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] PATH_INFO: `/v3/projects/c45f3c23c3234b078b72f10c08a78aec` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:55.849380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45f3c23c3234b078b72f10c08a78aec)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:55.851520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:55.960022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:55.978150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45f3c23c3234b078b72f10c08a78aec', 'request_id': 'req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:56.004694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45f3c23c3234b078b72f10c08a78aec', 'request_id': 'req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:56.004694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45f3c23c3234b078b72f10c08a78aec', 'request_id': 'req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:56.077393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] The token cache is being invalidate because project c45f3c23c3234b078b72f10c08a78aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:30:56.077684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:56.078329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7b6809ee-f7fc-42b1-8ca8-c6e908a52764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:30:56.079851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 958/1931] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:30:55 2022] DELETE /identity/v3/projects/c45f3c23c3234b078b72f10c08a78aec => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:56.084891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854edd-ff6e-4a5f-b077-577e10757140 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.085383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854edd-ff6e-4a5f-b077-577e10757140 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.085471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20854edd-ff6e-4a5f-b077-577e10757140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.109582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-20854edd-ff6e-4a5f-b077-577e10757140 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:56.110874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-20854edd-ff6e-4a5f-b077-577e10757140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:56.192553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 974/1932] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:56.201471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d5a5f-fb4d-4e03-8134-88ac970814b1 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.201781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d5a5f-fb4d-4e03-8134-88ac970814b1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.202063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d5a5f-fb4d-4e03-8134-88ac970814b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.224361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-194d5a5f-fb4d-4e03-8134-88ac970814b1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:56.225596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-194d5a5f-fb4d-4e03-8134-88ac970814b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:56.233844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 959/1933] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:56.238908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20854edd-ff6e-4a5f-b077-577e10757140 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.249879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20854edd-ff6e-4a5f-b077-577e10757140 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.254550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.258935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.258935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.258935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.258935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:30:56.258935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.258935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.269988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 975/1934] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:56.274319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d5a5f-fb4d-4e03-8134-88ac970814b1 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.283518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d5a5f-fb4d-4e03-8134-88ac970814b1 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.288657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.289284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.289564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.289836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.290482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.292394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.376464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 960/1935] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:56.380661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.390516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cb9e9-1be9-4127-ba44-869885f2d03e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.417236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:56.447451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.448119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.448419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.448968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.449863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.452361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.534875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 976/1936] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:56.539641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.549803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377beb2-8f22-42a7-bb83-93534ae0344e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.554401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.555026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.555304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.555572 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.556204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.557815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.567746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 961/1937] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:56.571631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.581356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b890-ebe9-4d7c-b639-5eac7c5932ce admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.586516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61181b17-2bf3-4f2d-8511-362914b73424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.587137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181b17-2bf3-4f2d-8511-362914b73424 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.587414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181b17-2bf3-4f2d-8511-362914b73424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.587680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181b17-2bf3-4f2d-8511-362914b73424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.588326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61181b17-2bf3-4f2d-8511-362914b73424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.589914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61181b17-2bf3-4f2d-8511-362914b73424 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.600313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 977/1938] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:56.604254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.613888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9160a5-432f-44f4-8ea6-943c98f7ee54 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.618748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.619369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.619644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.619913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] PATH_INFO: `/v3/projects/bc49003f8c5841eaa6388bf9bb49d4e4/users/85ed2022b6e244a3aa577da72137c28f/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.633624 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc49003f8c5841eaa6388bf9bb49d4e4, user_id=85ed2022b6e244a3aa577da72137c28f, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.636258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.688496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 962/1939] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:56 2022] PUT /identity/v3/projects/bc49003f8c5841eaa6388bf9bb49d4e4/users/85ed2022b6e244a3aa577da72137c28f/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:30:56.708846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13b63-f3f5-4187-a0f9-36416590bc3f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.709137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13b63-f3f5-4187-a0f9-36416590bc3f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.709416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be13b63-f3f5-4187-a0f9-36416590bc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.736241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6be13b63-f3f5-4187-a0f9-36416590bc3f None None] MFA Rules not processed for user `85ed2022b6e244a3aa577da72137c28f`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:56.737452 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6be13b63-f3f5-4187-a0f9-36416590bc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:56.802398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 978/1940] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:56.807055 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.867794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19727e77-f8a5-46d5-94b0-452bf85b0a3a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.872566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.873156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.873453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.873721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.874341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.876276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.969712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 963/1941] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:56.971092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13b63-f3f5-4187-a0f9-36416590bc3f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.982119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be13b63-f3f5-4187-a0f9-36416590bc3f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:57.048778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:57.049563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:57.049911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:57.050140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:57.051406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:57.053669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:57.133376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 979/1942] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:30:57.135286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:57.145634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9d0b9-2ab5-494d-9172-4631ecf6f18b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:57.150140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:57.150802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:57.151082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:57.151348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:57.152011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:57.153584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:57.164538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 964/1943] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:30:57.168603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:57.179657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04fa14-0f2b-4d34-b263-fbc8ea16855c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:57.184850 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:57.185514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:57.185797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:57.186073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] PATH_INFO: `/v3/projects/592e12937ef44b70b6ba11e364be36fd/users/c8a18d38dc74423dab319f474b52e664/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:57.205999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592e12937ef44b70b6ba11e364be36fd, user_id=c8a18d38dc74423dab319f474b52e664, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:57.208848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:57.300099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 980/1944] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:57 2022] PUT /identity/v3/projects/592e12937ef44b70b6ba11e364be36fd/users/c8a18d38dc74423dab319f474b52e664/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:30:57.304720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:57.314593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ce839-5d9e-4f1d-936c-cca2a86ad1cf admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:57.335255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:57.372529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf976f94-880b-4f08-84e0-5e26aeeee5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:57.373134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976f94-880b-4f08-84e0-5e26aeeee5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:57.373412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976f94-880b-4f08-84e0-5e26aeeee5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:57.373689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976f94-880b-4f08-84e0-5e26aeeee5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:57.374326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf976f94-880b-4f08-84e0-5e26aeeee5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:57.375963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf976f94-880b-4f08-84e0-5e26aeeee5d0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:57.387586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 965/1945] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:30:57.392253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:57.397892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:57.403392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a18049-7d27-4b6d-a94b-248bd509d991 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:57.408769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:57.409462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:57.410285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:57.416501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] PATH_INFO: `/v3/projects/592e12937ef44b70b6ba11e364be36fd/users/c8a18d38dc74423dab319f474b52e664/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:57.416501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=592e12937ef44b70b6ba11e364be36fd, user_id=c8a18d38dc74423dab319f474b52e664, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:57.419264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:57.435177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 981/1946] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:30:57 2022] PUT /identity/v3/projects/592e12937ef44b70b6ba11e364be36fd/users/c8a18d38dc74423dab319f474b52e664/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:30:57.445460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0f2d-28ff-4a22-ba19-463e2cac636d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:57.445751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0f2d-28ff-4a22-ba19-463e2cac636d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:57.446029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0f2d-28ff-4a22-ba19-463e2cac636d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:57.475344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2a4d0f2d-28ff-4a22-ba19-463e2cac636d None None] MFA Rules not processed for user `c8a18d38dc74423dab319f474b52e664`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:57.476631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2a4d0f2d-28ff-4a22-ba19-463e2cac636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:57.542613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 966/1947] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:57.553842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:57.555588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:57.611398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfd404-6f4b-4a9f-ba49-1161f70fc26f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:57.615788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f45bf2-c404-4ac2-8257-8c32527b8988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:57.616390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45bf2-c404-4ac2-8257-8c32527b8988 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:57.616648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45bf2-c404-4ac2-8257-8c32527b8988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:57.616913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45bf2-c404-4ac2-8257-8c32527b8988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:57.670073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f45bf2-c404-4ac2-8257-8c32527b8988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:57.671525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f45bf2-c404-4ac2-8257-8c32527b8988 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:57.680766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 982/1948] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:01.633433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92eba0-d0f3-43ef-88dd-7ede05014cc4 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:01.633837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92eba0-d0f3-43ef-88dd-7ede05014cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:01.633978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92eba0-d0f3-43ef-88dd-7ede05014cc4 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:01.635406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 967/1949] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:31:01.650631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f7a1-2e06-416a-abb8-f131e2ef2c4c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:01.655730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f7a1-2e06-416a-abb8-f131e2ef2c4c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:01.655730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f7a1-2e06-416a-abb8-f131e2ef2c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:01.674269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4619f7a1-2e06-416a-abb8-f131e2ef2c4c None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:01.679875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4619f7a1-2e06-416a-abb8-f131e2ef2c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:01.771159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 983/1950] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:01.843206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92eba0-d0f3-43ef-88dd-7ede05014cc4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:01.843206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3c92eba0-d0f3-43ef-88dd-7ede05014cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:01.867824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f7a1-2e06-416a-abb8-f131e2ef2c4c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:01.870775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92eba0-d0f3-43ef-88dd-7ede05014cc4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:01.876910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87b759-0198-4216-9660-b5009c30dcc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:01.877620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b759-0198-4216-9660-b5009c30dcc0 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:01.877961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b759-0198-4216-9660-b5009c30dcc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:01.878294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87b759-0198-4216-9660-b5009c30dcc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:01.893413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b759-0198-4216-9660-b5009c30dcc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:01.895921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87b759-0198-4216-9660-b5009c30dcc0 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:01.911148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 968/1951] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:01.917482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:01.952919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f7a1-2e06-416a-abb8-f131e2ef2c4c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:01.958168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:01.958921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:01.959432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:01.959907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] PATH_INFO: `/v3/users/85ed2022b6e244a3aa577da72137c28f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:01.961538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ed2022b6e244a3aa577da72137c28f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:01.963549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:02.170494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ed2022b6e244a3aa577da72137c28f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:02.171348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ed2022b6e244a3aa577da72137c28f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:02.243840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ed2022b6e244a3aa577da72137c28f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:02.256770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ed2022b6e244a3aa577da72137c28f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:02.257466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ed2022b6e244a3aa577da72137c28f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:02.270643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 984/1952] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:01 2022] DELETE /identity/v3/users/85ed2022b6e244a3aa577da72137c28f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:02.275227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:02.275508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:02.275776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:02.294998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:02.296247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:02.360078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 969/1953] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:02.370021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:02.371465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:02.438796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069d54-77a4-4706-97ea-a75ed30f69f3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:02.442383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:02.443028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:02.443307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:02.443573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:02.452953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:02.454408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:02.464285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 985/1954] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:02.896074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:02.898095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:02.920250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecec0d-3432-408f-a1a6-85845afd8c9a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:02.929736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:02.930251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:02.930543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:02.930767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] PATH_INFO: `/v3/projects/bc49003f8c5841eaa6388bf9bb49d4e4` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:02.934605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc49003f8c5841eaa6388bf9bb49d4e4)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:02.938845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:03.058489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:03.076028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc49003f8c5841eaa6388bf9bb49d4e4', 'request_id': 'req-5eadf517-1b83-4a47-ac14-4a9b2502dfea'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.082307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc49003f8c5841eaa6388bf9bb49d4e4', 'request_id': 'req-5eadf517-1b83-4a47-ac14-4a9b2502dfea'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.087904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc49003f8c5841eaa6388bf9bb49d4e4', 'request_id': 'req-5eadf517-1b83-4a47-ac14-4a9b2502dfea'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.164331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] The token cache is being invalidate because project bc49003f8c5841eaa6388bf9bb49d4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:03.164615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.165229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.166747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 970/1955] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:02 2022] DELETE /identity/v3/projects/bc49003f8c5841eaa6388bf9bb49d4e4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:03.171091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:03.191575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022d520-bc6a-42db-8952-3d806832b6f3 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:03.271688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:03.280440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:03.282669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:03.288934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:03.288934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] PATH_INFO: `/v3/users/c8a18d38dc74423dab319f474b52e664` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:03.290472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a18d38dc74423dab319f474b52e664)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:03.292622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:03.483590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a18d38dc74423dab319f474b52e664'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.484257 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a18d38dc74423dab319f474b52e664'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.555235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a18d38dc74423dab319f474b52e664'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.568594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a18d38dc74423dab319f474b52e664'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.569362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7077c9d9-2251-4e1e-a2fd-f248df2f4acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a18d38dc74423dab319f474b52e664'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:03.582216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 986/1956] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:03 2022] DELETE /identity/v3/users/c8a18d38dc74423dab319f474b52e664 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:31:03.980003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:03.981525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:03.995951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadf517-1b83-4a47-ac14-4a9b2502dfea admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:04.054151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:04.054813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.055090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.055362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] PATH_INFO: `/v3/projects/592e12937ef44b70b6ba11e364be36fd` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.056504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=592e12937ef44b70b6ba11e364be36fd)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:04.058453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:04.178190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592e12937ef44b70b6ba11e364be36fd', 'request_id': 'req-d85d4cec-5862-484e-9c89-07b4e4372d57'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:04.185935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592e12937ef44b70b6ba11e364be36fd', 'request_id': 'req-d85d4cec-5862-484e-9c89-07b4e4372d57'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:04.192977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592e12937ef44b70b6ba11e364be36fd', 'request_id': 'req-d85d4cec-5862-484e-9c89-07b4e4372d57'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:04.259330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] The token cache is being invalidate because project 592e12937ef44b70b6ba11e364be36fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:04.259624 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:04.260314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d85d4cec-5862-484e-9c89-07b4e4372d57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:04.262043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 971/1957] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:03 2022] DELETE /identity/v3/projects/592e12937ef44b70b6ba11e364be36fd => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:04.272344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde57b-5d5b-4cc5-bb5f-f05c5a3b35ed None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.272852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde57b-5d5b-4cc5-bb5f-f05c5a3b35ed None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.273115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde57b-5d5b-4cc5-bb5f-f05c5a3b35ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.292540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c2dde57b-5d5b-4cc5-bb5f-f05c5a3b35ed None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:04.296088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c2dde57b-5d5b-4cc5-bb5f-f05c5a3b35ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:04.370218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 987/1958] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:04.379596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdb21e-2d91-4615-8f1f-7008778426aa None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.379871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdb21e-2d91-4615-8f1f-7008778426aa None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.380141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdb21e-2d91-4615-8f1f-7008778426aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.401022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d7fdb21e-2d91-4615-8f1f-7008778426aa None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:04.402262 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d7fdb21e-2d91-4615-8f1f-7008778426aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:04.412836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 972/1959] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:04.417480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dde57b-5d5b-4cc5-bb5f-f05c5a3b35ed None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:04.432206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dde57b-5d5b-4cc5-bb5f-f05c5a3b35ed None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:04.437142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:04.437757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.438037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.438309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.439006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:31:04.439305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:04.441462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:04.453734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 988/1960] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:31:04.459694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdb21e-2d91-4615-8f1f-7008778426aa None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:04.470156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdb21e-2d91-4615-8f1f-7008778426aa None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:04.475754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:04.476701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.477048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.477391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.478534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:04.480552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:04.594016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 973/1961] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:04 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:04.598843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:04.609042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca759b-5b7a-48f3-a8bd-4ab08f233df0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:04.667300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:04.667930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.668213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.668514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.669137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:04.671117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:04.681163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:04.786773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 989/1962] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:04 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:04.790883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:04.801463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e0647-d079-4e09-bf51-2468e75b62fc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:04.805912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:04.806490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.806806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.807076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.807731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:04.809309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:04.819547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 974/1963] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:04.830507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:04.840712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ad652-005f-4875-8cc1-7954701c7cbe admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:04.845658 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bcc25-3dcd-483b-8aa7-b649f45dcc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:04.846247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bcc25-3dcd-483b-8aa7-b649f45dcc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.846521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bcc25-3dcd-483b-8aa7-b649f45dcc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.846831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bcc25-3dcd-483b-8aa7-b649f45dcc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.847460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bcc25-3dcd-483b-8aa7-b649f45dcc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:04.849044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bcc25-3dcd-483b-8aa7-b649f45dcc5a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:04.859374 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 990/1964] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:04.863963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:04.874049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1ed8f-3650-4de9-821e-c9596e4fb85f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:04.879114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:04.879745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.880019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.880284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] PATH_INFO: `/v3/projects/d712579d3d9a45c18768b52cee48cbd9/users/3ab1e44ca181489b8a4b19ef7fda9844/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:04.882420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:04.895307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d712579d3d9a45c18768b52cee48cbd9, user_id=3ab1e44ca181489b8a4b19ef7fda9844, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:04.897975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:04.966567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 975/1965] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:04 2022] PUT /identity/v3/projects/d712579d3d9a45c18768b52cee48cbd9/users/3ab1e44ca181489b8a4b19ef7fda9844/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:04.977253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:04.977555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:04.977807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:05.005319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] MFA Rules not processed for user `3ab1e44ca181489b8a4b19ef7fda9844`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:05.006523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:05.068460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 991/1966] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:05.079383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:05.081120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:05.136250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47457e29-08aa-4590-99c2-c3e07236bd51 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:05.139765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d219d-ef32-4136-a232-770bb8f0122b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:05.140349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d219d-ef32-4136-a232-770bb8f0122b service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:05.140623 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d219d-ef32-4136-a232-770bb8f0122b service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:05.140888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d219d-ef32-4136-a232-770bb8f0122b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:05.148901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d219d-ef32-4136-a232-770bb8f0122b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:05.150317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d219d-ef32-4136-a232-770bb8f0122b service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:05.158842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 976/1967] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:15.065549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:15.155058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:24.735282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:24.736728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:24.831843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1ff1-7622-409c-8ff7-9d66484d0a9e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:24.836446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:24.837717 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:24.837996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:24.838259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] PATH_INFO: `/v3/users/9d9cc1c457d1419b9ef2f7d7b723ef37` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:24.839435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9cc1c457d1419b9ef2f7d7b723ef37)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:24.841391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:25.079762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9cc1c457d1419b9ef2f7d7b723ef37'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:25.080461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9cc1c457d1419b9ef2f7d7b723ef37'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:25.157193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9cc1c457d1419b9ef2f7d7b723ef37'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:25.170775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9cc1c457d1419b9ef2f7d7b723ef37'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:25.171570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9cc1c457d1419b9ef2f7d7b723ef37'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:25.185539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 992/1968] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:24 2022] DELETE /identity/v3/users/9d9cc1c457d1419b9ef2f7d7b723ef37 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:25.190153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:25.190628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:25.190749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:25.209441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:25.210608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:25.268542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 977/1969] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:25.279095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:25.280539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:25.337249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b65c9-b0e2-46b7-9aa9-00dc622326e5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:25.341670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:25.342309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:25.345219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:25.345219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:25.352135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:25.353643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:25.362917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 993/1970] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:25.868032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:25.870181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:25.889723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a43c1-9d21-498d-b74c-2d37a4e6bbb4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:25.902563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:25.903366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:25.903749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:25.904091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] PATH_INFO: `/v3/projects/0450a0f589a74fcc82f8335c4a699db0` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:25.906130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0450a0f589a74fcc82f8335c4a699db0)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:25.911681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:26.112789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0450a0f589a74fcc82f8335c4a699db0', 'request_id': 'req-a24e364b-81af-4c09-8b1b-522915039706'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.120892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0450a0f589a74fcc82f8335c4a699db0', 'request_id': 'req-a24e364b-81af-4c09-8b1b-522915039706'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.131333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0450a0f589a74fcc82f8335c4a699db0', 'request_id': 'req-a24e364b-81af-4c09-8b1b-522915039706'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.243249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] The token cache is being invalidate because project 0450a0f589a74fcc82f8335c4a699db0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:26.247999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.247999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.247999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 978/1971] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:25 2022] DELETE /identity/v3/projects/0450a0f589a74fcc82f8335c4a699db0 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:26.249769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:26.259496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d674de-5d37-4c68-a11e-27f02d199ace service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:26.279179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:26.317912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:26.318467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:26.318785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:26.319061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] PATH_INFO: `/v3/users/19c5bbb335574bb0ba413d5492669f8b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:26.320169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c5bbb335574bb0ba413d5492669f8b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:26.322097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:26.542914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c5bbb335574bb0ba413d5492669f8b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.543569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c5bbb335574bb0ba413d5492669f8b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.602983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c5bbb335574bb0ba413d5492669f8b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.616849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c5bbb335574bb0ba413d5492669f8b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.617600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2566a28b-723c-464d-84c0-1aa5a01e8d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c5bbb335574bb0ba413d5492669f8b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:26.629881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 994/1972] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:26 2022] DELETE /identity/v3/users/19c5bbb335574bb0ba413d5492669f8b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:27.181567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:27.183352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:27.199334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e364b-81af-4c09-8b1b-522915039706 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:27.268359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:27.268922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:27.269092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:27.269581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] PATH_INFO: `/v3/projects/fd9d1b918bef43378d63d0a3c5f9b1e4` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:27.271236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9d1b918bef43378d63d0a3c5f9b1e4)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:27.273075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:27.408617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:27.435839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9d1b918bef43378d63d0a3c5f9b1e4', 'request_id': 'req-6a5fda53-82ce-4ab6-b600-88aeeaec5869'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:27.442859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9d1b918bef43378d63d0a3c5f9b1e4', 'request_id': 'req-6a5fda53-82ce-4ab6-b600-88aeeaec5869'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:27.448847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9d1b918bef43378d63d0a3c5f9b1e4', 'request_id': 'req-6a5fda53-82ce-4ab6-b600-88aeeaec5869'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:27.557772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] The token cache is being invalidate because project fd9d1b918bef43378d63d0a3c5f9b1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:27.558059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:27.562350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6a5fda53-82ce-4ab6-b600-88aeeaec5869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:27.562350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 979/1973] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:27 2022] DELETE /identity/v3/projects/fd9d1b918bef43378d63d0a3c5f9b1e4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:27.565239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1eaad-040b-4716-a334-8d5b9e563ae6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:27.565707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1eaad-040b-4716-a334-8d5b9e563ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:27.565910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1eaad-040b-4716-a334-8d5b9e563ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:27.587378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-63f1eaad-040b-4716-a334-8d5b9e563ae6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:27.590262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-63f1eaad-040b-4716-a334-8d5b9e563ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:27.648611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 995/1974] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:27.656835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ab327-670d-4547-989d-2c4ec729887b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:27.657122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ab327-670d-4547-989d-2c4ec729887b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:27.657398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ab327-670d-4547-989d-2c4ec729887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:27.679808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-913ab327-670d-4547-989d-2c4ec729887b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:27.681119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-913ab327-670d-4547-989d-2c4ec729887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:27.690378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 980/1975] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:27.694614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1eaad-040b-4716-a334-8d5b9e563ae6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:27.705552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1eaad-040b-4716-a334-8d5b9e563ae6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:27.710504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:27.711164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:27.711439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:27.711706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:27.712871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:31:27.713161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:27.715345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:27.732329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 996/1976] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:31:27.734608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ab327-670d-4547-989d-2c4ec729887b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:27.749518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ab327-670d-4547-989d-2c4ec729887b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:27.754670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:27.755302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:27.755587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:27.755919 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:27.756556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:27.758513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:27.868369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 981/1977] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:27 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:27.871774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:27.893497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9bfce-bf81-4d03-9ef5-2645725a9d9c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:27.930530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:27.992830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:27.994187 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:27.994468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:27.994757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:27.995383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:27.997304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:28.098971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 997/1978] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:27 2022] POST /identity/v3/users => generated 284 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:28.100217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:28.112867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a077719-d373-4bee-aaa2-c761dfeea9d3 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:28.119469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:28.120312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.120671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.121074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.121929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:28.123877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:28.143372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 982/1979] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:31:28.144656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:28.161740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393c524-5e3a-43d9-bdee-c2bf8ceeb266 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:28.164522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae89b32-46ce-46d6-a0b8-f911819effe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:28.165851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae89b32-46ce-46d6-a0b8-f911819effe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.166135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae89b32-46ce-46d6-a0b8-f911819effe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.166396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae89b32-46ce-46d6-a0b8-f911819effe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.167857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae89b32-46ce-46d6-a0b8-f911819effe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:28.170322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae89b32-46ce-46d6-a0b8-f911819effe1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:28.197480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 998/1980] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:28.206407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:28.224977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a846ae6-80be-4978-9413-0d27e64f57e9 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:28.224977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:28.224977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.225289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.225444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] PATH_INFO: `/v3/projects/b571d4335ccc4cc9aa50ba220ab0b547/users/d4fe5493fae145a79848e47683229c0b/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.243954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b571d4335ccc4cc9aa50ba220ab0b547, user_id=d4fe5493fae145a79848e47683229c0b, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:28.246613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:28.324531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 983/1981] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:28 2022] PUT /identity/v3/projects/b571d4335ccc4cc9aa50ba220ab0b547/users/d4fe5493fae145a79848e47683229c0b/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:28.336105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173feac-c922-4c9f-9e8b-2e741ef1eb7c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.336393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173feac-c922-4c9f-9e8b-2e741ef1eb7c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.336674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173feac-c922-4c9f-9e8b-2e741ef1eb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.369506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-5173feac-c922-4c9f-9e8b-2e741ef1eb7c None None] MFA Rules not processed for user `d4fe5493fae145a79848e47683229c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:28.370833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5173feac-c922-4c9f-9e8b-2e741ef1eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:28.456979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 999/1982] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:31:28.461759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:28.534926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6a480-3dc8-499d-ad71-02be573c4042 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:28.539967 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:28.541197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.541480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.541757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.542669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:28.544774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:28.644939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 984/1983] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:28 2022] POST /identity/v3/projects => generated 367 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:31:28.649862 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173feac-c922-4c9f-9e8b-2e741ef1eb7c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:28.661454 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173feac-c922-4c9f-9e8b-2e741ef1eb7c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:28.727471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:28.728887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.729211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.729668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.730304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:28.732242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:28.867280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1000/1984] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:28 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:31:28.871472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:28.881980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027ad71-09a8-4a54-a92f-6e0741ffadd2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:28.887554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:28.888174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.888438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.888711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.889875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:28.891171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:28.910061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:28.916278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 985/1985] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:31:28.921867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd6a13-f7ac-42b8-b076-4e4d7929a4ee admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:28.927514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:28.929624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:28.929624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:28.929624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] PATH_INFO: `/v3/projects/5e4a677e63a547dbb16cab36e177a52c/users/fe135b8cf2fe48bd94344ca1e0042e87/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:28.949579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4a677e63a547dbb16cab36e177a52c, user_id=fe135b8cf2fe48bd94344ca1e0042e87, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:28.954086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:29.056971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1001/1986] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:28 2022] PUT /identity/v3/projects/5e4a677e63a547dbb16cab36e177a52c/users/fe135b8cf2fe48bd94344ca1e0042e87/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:29.063923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:29.078308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681e1d7-0b27-487a-b4be-c8cceb44baed admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:29.109764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:29.158045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a6b50-b918-4a0c-8a11-b63919f147f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:29.158714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6b50-b918-4a0c-8a11-b63919f147f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:29.158998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6b50-b918-4a0c-8a11-b63919f147f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:29.159272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6b50-b918-4a0c-8a11-b63919f147f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:29.159917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a6b50-b918-4a0c-8a11-b63919f147f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:29.161541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a6b50-b918-4a0c-8a11-b63919f147f2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:29.175655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 986/1987] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:29.179958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:29.190218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:29.201130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ce5e5-d218-4d69-a3dc-e2ffe714ad0b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:29.217290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:29.221234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:29.221640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:29.222393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] PATH_INFO: `/v3/projects/5e4a677e63a547dbb16cab36e177a52c/users/fe135b8cf2fe48bd94344ca1e0042e87/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:29.228499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4a677e63a547dbb16cab36e177a52c, user_id=fe135b8cf2fe48bd94344ca1e0042e87, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:29.234081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:29.250414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1002/1988] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:29 2022] PUT /identity/v3/projects/5e4a677e63a547dbb16cab36e177a52c/users/fe135b8cf2fe48bd94344ca1e0042e87/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:31:29.261689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:29.262307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:29.262666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:29.307397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] MFA Rules not processed for user `fe135b8cf2fe48bd94344ca1e0042e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:29.309140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:29.405110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 987/1989] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:29.427983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:29.434472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:29.539678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcbf22-4308-4d8b-8e4e-c64685333eb8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:29.544715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:29.545327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:29.545602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:29.545877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:29.557089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:29.558525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:29.579849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1003/1990] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:29.666596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:29.668064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:29.682215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e699b6a-4a4f-4ca9-8343-97ad68083610 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:29.686230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbb226-dd1b-4d78-966b-8cc899f8f1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:29.687018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbb226-dd1b-4d78-966b-8cc899f8f1dc service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:29.687537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbb226-dd1b-4d78-966b-8cc899f8f1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:29.688043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbb226-dd1b-4d78-966b-8cc899f8f1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:29.751586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbb226-dd1b-4d78-966b-8cc899f8f1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:29.753032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbb226-dd1b-4d78-966b-8cc899f8f1dc service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:29.763222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 988/1991] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:34.512179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:34.574584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c93db-6288-475d-a23b-297d5d0bf033 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:34.579753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:34.580637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:34.581006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:34.581366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] PATH_INFO: `/v3/users/d4fe5493fae145a79848e47683229c0b` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:34.582973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fe5493fae145a79848e47683229c0b)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:34.584941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:34.739951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fe5493fae145a79848e47683229c0b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:34.740607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fe5493fae145a79848e47683229c0b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:34.795834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fe5493fae145a79848e47683229c0b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:34.809020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fe5493fae145a79848e47683229c0b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:34.809694 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fe5493fae145a79848e47683229c0b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:34.822763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1004/1992] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:34 2022] DELETE /identity/v3/users/d4fe5493fae145a79848e47683229c0b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:34.827156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:34.827609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:34.827720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:34.847242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:34.848441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:34.907149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:34.925642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 989/1993] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:34.935568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:34.936975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:34.950970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:34.994840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122dc9a-211d-4577-9fc3-e096b4fe21d7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:34.998406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:34.999026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:34.999303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:34.999568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:35.008411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:35.010752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:35.019961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1005/1994] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:35.529300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:35.531460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:35.545263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138e0a-dfe9-416b-8ca2-1a7ad674e644 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:35.549881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:35.555163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:35.555163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:35.555163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] PATH_INFO: `/v3/projects/b571d4335ccc4cc9aa50ba220ab0b547` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:35.555163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b571d4335ccc4cc9aa50ba220ab0b547)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:35.555163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:35.655241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b571d4335ccc4cc9aa50ba220ab0b547', 'request_id': 'req-d620479d-bc4c-4e38-ab56-926472424141'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:35.661337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b571d4335ccc4cc9aa50ba220ab0b547', 'request_id': 'req-d620479d-bc4c-4e38-ab56-926472424141'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:35.667423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b571d4335ccc4cc9aa50ba220ab0b547', 'request_id': 'req-d620479d-bc4c-4e38-ab56-926472424141'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:35.760162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] The token cache is being invalidate because project b571d4335ccc4cc9aa50ba220ab0b547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:35.760451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:35.761081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:35.762589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 990/1995] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:35 2022] DELETE /identity/v3/projects/b571d4335ccc4cc9aa50ba220ab0b547 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:35.766707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:35.776492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44abbc1-a090-4acd-bdb5-228da6916512 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:35.857247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:35.857881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:35.858159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:35.858434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] PATH_INFO: `/v3/users/fe135b8cf2fe48bd94344ca1e0042e87` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:35.859589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe135b8cf2fe48bd94344ca1e0042e87)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:35.861499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:36.091164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe135b8cf2fe48bd94344ca1e0042e87'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.091818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe135b8cf2fe48bd94344ca1e0042e87'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.168375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe135b8cf2fe48bd94344ca1e0042e87'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.188155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe135b8cf2fe48bd94344ca1e0042e87'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.189066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c1344129-d072-45a7-ae11-6ae5358f79ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe135b8cf2fe48bd94344ca1e0042e87'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.201755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1006/1996] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:35 2022] DELETE /identity/v3/users/fe135b8cf2fe48bd94344ca1e0042e87 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:36.611722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:36.613660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:36.630226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620479d-bc4c-4e38-ab56-926472424141 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:36.690047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:36.690587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:36.690900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:36.691170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] PATH_INFO: `/v3/projects/5e4a677e63a547dbb16cab36e177a52c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:36.692269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4a677e63a547dbb16cab36e177a52c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:36.694152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:36.793773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4a677e63a547dbb16cab36e177a52c', 'request_id': 'req-b18ff11f-5752-4760-9511-c90c95d65cf1'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.800071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4a677e63a547dbb16cab36e177a52c', 'request_id': 'req-b18ff11f-5752-4760-9511-c90c95d65cf1'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.805577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4a677e63a547dbb16cab36e177a52c', 'request_id': 'req-b18ff11f-5752-4760-9511-c90c95d65cf1'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.869944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:36.874350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] The token cache is being invalidate because project 5e4a677e63a547dbb16cab36e177a52c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:36.874639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.875279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b18ff11f-5752-4760-9511-c90c95d65cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:36.876762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 991/1997] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:36 2022] DELETE /identity/v3/projects/5e4a677e63a547dbb16cab36e177a52c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:36.887975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd477d64-203a-4e21-84c8-f6af41ab5f51 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:36.888447 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd477d64-203a-4e21-84c8-f6af41ab5f51 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:36.888537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd477d64-203a-4e21-84c8-f6af41ab5f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:36.908668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-cd477d64-203a-4e21-84c8-f6af41ab5f51 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:36.909862 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-cd477d64-203a-4e21-84c8-f6af41ab5f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:36.939145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:36.968783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1007/1998] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:36.976031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4d04b-bc7b-4b3d-8774-ecbed69e7512 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:36.976319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4d04b-bc7b-4b3d-8774-ecbed69e7512 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:36.976596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4d04b-bc7b-4b3d-8774-ecbed69e7512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:36.997054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ecc4d04b-bc7b-4b3d-8774-ecbed69e7512 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:36.998316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ecc4d04b-bc7b-4b3d-8774-ecbed69e7512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:37.006135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 992/1999] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:37.010882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd477d64-203a-4e21-84c8-f6af41ab5f51 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.020622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd477d64-203a-4e21-84c8-f6af41ab5f51 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.025892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.026549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.026856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.027124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.027760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:31:37.028054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.029624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.040843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1008/2000] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:31:37.045014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4d04b-bc7b-4b3d-8774-ecbed69e7512 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.054209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4d04b-bc7b-4b3d-8774-ecbed69e7512 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.058928 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.059525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.059826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.060100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.060733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.062734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.173512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 993/2001] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:37 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:31:37.177661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.188270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3765a-89fb-4b4b-9501-894a2cbc2845 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.245861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.246467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.246771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.247045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.247740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.249725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.351531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1009/2002] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:37 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:37.356323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.366915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69256e21-8545-4e95-a55e-e0bd8b2f9f97 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.371962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.372551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.372824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.373090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.373725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.375355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.384936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 994/2003] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:37.388875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.398875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865ac1c-9b4b-473c-b027-1358f6de312d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.404210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f61cfe-c6e2-41f1-95bc-a712140df84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.404808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f61cfe-c6e2-41f1-95bc-a712140df84f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.405083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f61cfe-c6e2-41f1-95bc-a712140df84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.405351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f61cfe-c6e2-41f1-95bc-a712140df84f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.405974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f61cfe-c6e2-41f1-95bc-a712140df84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.407580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f61cfe-c6e2-41f1-95bc-a712140df84f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.422072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1010/2004] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:37.427854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.438117 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5c226-a484-476b-bc9f-5a768ca3e37f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.442626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.443300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.443578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.443847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] PATH_INFO: `/v3/projects/d2bb400f81244c0391bf26bbe9ef71e6/users/c33295211b3646b5b955da7036c8d4d5/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.458859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2bb400f81244c0391bf26bbe9ef71e6, user_id=c33295211b3646b5b955da7036c8d4d5, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.461509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.534095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 995/2005] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:37 2022] PUT /identity/v3/projects/d2bb400f81244c0391bf26bbe9ef71e6/users/c33295211b3646b5b955da7036c8d4d5/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:37.545919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38df08-05f5-4302-8716-7e9f6c6f7a9a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.546186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38df08-05f5-4302-8716-7e9f6c6f7a9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.546486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38df08-05f5-4302-8716-7e9f6c6f7a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.574128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2a38df08-05f5-4302-8716-7e9f6c6f7a9a None None] MFA Rules not processed for user `c33295211b3646b5b955da7036c8d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:37.575348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2a38df08-05f5-4302-8716-7e9f6c6f7a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:37.638437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1011/2006] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:37.642932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.718880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985ba43-a62b-4382-aebb-4c629a248a81 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.723602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.724205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.724482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.724752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.725380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.727321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.810539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 996/2007] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:37 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:37.814394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38df08-05f5-4302-8716-7e9f6c6f7a9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.825566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38df08-05f5-4302-8716-7e9f6c6f7a9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.851530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:37.881747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.882534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.883068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.883694 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.884395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.886309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.962455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1012/2008] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:37 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:31:37.966459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:37.976627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cb623-c0cd-41d7-b975-09960484869c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:37.981647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:37.982223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:37.982496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:37.982802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:37.983427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:37.984996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:37.996213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 997/2009] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:38.000284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:38.010678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59401d-a8a3-4851-b150-e630ea83b90c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:38.015233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:38.015875 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:38.016160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:38.016433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] PATH_INFO: `/v3/projects/f476272449154715a42314be6ef2a169/users/48289ac2a1384701b8cdbb7c6dac381c/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:38.031348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f476272449154715a42314be6ef2a169, user_id=48289ac2a1384701b8cdbb7c6dac381c, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:38.033978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:38.099616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1013/2010] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:37 2022] PUT /identity/v3/projects/f476272449154715a42314be6ef2a169/users/48289ac2a1384701b8cdbb7c6dac381c/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:38.103958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:38.113710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d24d9-afa6-44a3-866e-3f3b3f7b17db admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:38.171407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6f79e-5c2f-46fa-be7a-8e822b26fa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:38.172003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6f79e-5c2f-46fa-be7a-8e822b26fa7c admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:38.172285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6f79e-5c2f-46fa-be7a-8e822b26fa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:38.172555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6f79e-5c2f-46fa-be7a-8e822b26fa7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:38.173185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6f79e-5c2f-46fa-be7a-8e822b26fa7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:38.174828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6f79e-5c2f-46fa-be7a-8e822b26fa7c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:38.184389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 998/2011] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:38.188485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:38.198675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc360a-7aa0-4662-881a-473807e7607a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:38.203218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:38.203855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:38.204131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:38.204404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] PATH_INFO: `/v3/projects/f476272449154715a42314be6ef2a169/users/48289ac2a1384701b8cdbb7c6dac381c/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:38.206290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f476272449154715a42314be6ef2a169, user_id=48289ac2a1384701b8cdbb7c6dac381c, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:38.208942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:38.219890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1014/2012] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:38 2022] PUT /identity/v3/projects/f476272449154715a42314be6ef2a169/users/48289ac2a1384701b8cdbb7c6dac381c/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:38.230475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22274b1e-1a87-43eb-b4e9-cf4efc476a4d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:38.230794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22274b1e-1a87-43eb-b4e9-cf4efc476a4d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:38.231071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22274b1e-1a87-43eb-b4e9-cf4efc476a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:38.239706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:38.260280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-22274b1e-1a87-43eb-b4e9-cf4efc476a4d None None] MFA Rules not processed for user `48289ac2a1384701b8cdbb7c6dac381c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:38.261540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-22274b1e-1a87-43eb-b4e9-cf4efc476a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:38.327504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 999/2013] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:38.339195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:38.340863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:38.397698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaeef5-f69b-40ae-938d-7b480be35fe1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:38.401867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:38.402460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:38.402766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:38.403037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:38.411837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:38.413264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:38.423548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1015/2014] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:38.633889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22274b1e-1a87-43eb-b4e9-cf4efc476a4d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:38.644185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22274b1e-1a87-43eb-b4e9-cf4efc476a4d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:38.647824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe8f17-467c-421a-9d70-6184f7947a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:38.648354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe8f17-467c-421a-9d70-6184f7947a2b service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:38.648629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe8f17-467c-421a-9d70-6184f7947a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:38.648899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe8f17-467c-421a-9d70-6184f7947a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:38.710565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe8f17-467c-421a-9d70-6184f7947a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:38.712016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe8f17-467c-421a-9d70-6184f7947a2b service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:38.721613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1000/2015] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:43.734664 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:43.736754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:43.814701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f95fd-3e94-4bb0-8b9d-2c1582788497 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:43.819414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:43.820122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:43.820461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:43.820809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] PATH_INFO: `/v3/users/995a0d3db89147b1b9749338acb2f5b5` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:43.822033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=995a0d3db89147b1b9749338acb2f5b5)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:43.824043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:43.935202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:44.039811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '995a0d3db89147b1b9749338acb2f5b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:44.041244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '995a0d3db89147b1b9749338acb2f5b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:44.160857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '995a0d3db89147b1b9749338acb2f5b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:44.174686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '995a0d3db89147b1b9749338acb2f5b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:44.175391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '995a0d3db89147b1b9749338acb2f5b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:44.195129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1016/2016] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:43 2022] DELETE /identity/v3/users/995a0d3db89147b1b9749338acb2f5b5 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:31:44.197854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:44.198332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:44.198588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:44.219685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:44.220933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:44.289744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1001/2017] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:31:44.300204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:44.301609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:44.359605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483444f-fb74-4182-9ff6-233114542d4d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:44.363224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31312840-c886-4de4-a195-98009c54a28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:44.363841 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312840-c886-4de4-a195-98009c54a28e service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:44.364122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312840-c886-4de4-a195-98009c54a28e service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:44.364767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31312840-c886-4de4-a195-98009c54a28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:44.376996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31312840-c886-4de4-a195-98009c54a28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:44.378596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31312840-c886-4de4-a195-98009c54a28e service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:44.406995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1017/2018] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:31:45.016647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:45.016647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:45.024932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a5f38-39e3-4854-ab72-bc4040b642fe None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:45.031903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:45.032563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.032847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.033130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] PATH_INFO: `/v3/projects/52c77ee5482845aaad46d0b6ecedd525` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.035292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c77ee5482845aaad46d0b6ecedd525)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:45.037434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:45.082463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:45.199151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c77ee5482845aaad46d0b6ecedd525', 'request_id': 'req-70de7898-4aad-4b66-9ca4-483fc242d949'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:45.208732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c77ee5482845aaad46d0b6ecedd525', 'request_id': 'req-70de7898-4aad-4b66-9ca4-483fc242d949'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:45.220121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c77ee5482845aaad46d0b6ecedd525', 'request_id': 'req-70de7898-4aad-4b66-9ca4-483fc242d949'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:45.295048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] The token cache is being invalidate because project 52c77ee5482845aaad46d0b6ecedd525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:45.295386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:45.296217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70de7898-4aad-4b66-9ca4-483fc242d949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:45.297896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1002/2019] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:45 2022] DELETE /identity/v3/projects/52c77ee5482845aaad46d0b6ecedd525 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:45.302910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100ad0a-484f-4f6c-a4c2-3eac5bd9d848 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.303411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100ad0a-484f-4f6c-a4c2-3eac5bd9d848 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.324968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100ad0a-484f-4f6c-a4c2-3eac5bd9d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.327596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3100ad0a-484f-4f6c-a4c2-3eac5bd9d848 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:45.328874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3100ad0a-484f-4f6c-a4c2-3eac5bd9d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:45.389876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1018/2020] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:31:45.397832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d825387-0ac0-41da-bec7-7edaf12c00a9 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.398125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d825387-0ac0-41da-bec7-7edaf12c00a9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.398405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d825387-0ac0-41da-bec7-7edaf12c00a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.429327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0d825387-0ac0-41da-bec7-7edaf12c00a9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:45.431145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0d825387-0ac0-41da-bec7-7edaf12c00a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:45.441883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1003/2021] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:45.445811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100ad0a-484f-4f6c-a4c2-3eac5bd9d848 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:45.458711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100ad0a-484f-4f6c-a4c2-3eac5bd9d848 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:45.461184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:45.461780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.462057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.462324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.463024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:31:45.463320 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:45.464967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:45.479355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1019/2022] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:31:45.484778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d825387-0ac0-41da-bec7-7edaf12c00a9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:45.495474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d825387-0ac0-41da-bec7-7edaf12c00a9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:45.500399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:45.500989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.501264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.501534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.502152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:45.504066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:45.612577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1004/2023] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:45 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:45.616733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:45.628645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69c9f-7ee0-4b6e-b83d-1bea8d581604 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:45.683618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:45.721863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:45.722566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.722939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.723282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.724615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:45.726983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:45.875255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1020/2024] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:45 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:45.876436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:45.887130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa38785-0de6-46a0-af05-625fda12af8a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:45.891670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:45.894518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.894518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.894518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.894518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:45.895172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:45.905878 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1005/2025] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:45.909890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:45.920840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e7ede-7323-4203-86f9-c08c3ef17034 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:45.925590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9c7b1-587d-4eb9-be97-8502aed03a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:45.926179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9c7b1-587d-4eb9-be97-8502aed03a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.926563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9c7b1-587d-4eb9-be97-8502aed03a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.926909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9c7b1-587d-4eb9-be97-8502aed03a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.927540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9c7b1-587d-4eb9-be97-8502aed03a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:45.929156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9c7b1-587d-4eb9-be97-8502aed03a93 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:45.947367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1021/2026] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:45.948442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:45.958776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8579e6a-b40a-4048-9196-5a5523ac1fc3 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:45.964052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:45.964706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:45.964988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:45.965264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] PATH_INFO: `/v3/projects/230b48971bab4b299b87a633641f7b75/users/530cbd6b8b3545ee85e83193749b2ecd/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:45.980653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=230b48971bab4b299b87a633641f7b75, user_id=530cbd6b8b3545ee85e83193749b2ecd, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:45.983345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:46.064375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1006/2027] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:45 2022] PUT /identity/v3/projects/230b48971bab4b299b87a633641f7b75/users/530cbd6b8b3545ee85e83193749b2ecd/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:46.076651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:46.076938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:46.077211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:46.106016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] MFA Rules not processed for user `530cbd6b8b3545ee85e83193749b2ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:46.106707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:46.172776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1022/2028] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:46.185832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:46.186069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:46.245844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1c0b7-66f9-4ecc-9062-560779b96ad6 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:46.251642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7beb33c-ae71-4d89-95e5-f7d4f12d99e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f33e1c573c044f086f99eb128c8f726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:46.252268 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb33c-ae71-4d89-95e5-f7d4f12d99e6 service glance] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:46.252576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb33c-ae71-4d89-95e5-f7d4f12d99e6 service glance] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:46.252894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beb33c-ae71-4d89-95e5-f7d4f12d99e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:46.263300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb33c-ae71-4d89-95e5-f7d4f12d99e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:46.264907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7beb33c-ae71-4d89-95e5-f7d4f12d99e6 service glance] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:46.274008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1007/2029] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:48.314364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:48.314996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:48.379196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442c9b4-a4f5-41c0-8a28-260f02aa1bf4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:48.383701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:48.384387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:48.384941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:48.384941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] PATH_INFO: `/v3/users/c33295211b3646b5b955da7036c8d4d5` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:48.386062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c33295211b3646b5b955da7036c8d4d5)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:48.388543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:48.651870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:48.654335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c33295211b3646b5b955da7036c8d4d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:48.655015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c33295211b3646b5b955da7036c8d4d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:48.761101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c33295211b3646b5b955da7036c8d4d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:48.773801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c33295211b3646b5b955da7036c8d4d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:48.774435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c33295211b3646b5b955da7036c8d4d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:48.786160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1023/2030] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:48 2022] DELETE /identity/v3/users/c33295211b3646b5b955da7036c8d4d5 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:48.790716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:48.791214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:48.791305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:48.811174 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:48.812396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:48.857942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:48.878927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1008/2031] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:48.886040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:48.888051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:48.953404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c82313-46cf-4604-a3af-04ee54620684 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:48.957144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:48.957755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:48.958029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:48.958297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:48.969579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:48.971464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:48.990720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1024/2032] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:49.445273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:49.446988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:49.459049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4143a-a88a-4af2-9955-1b238e070fe3 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:49.463631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:49.464344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:49.465125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:49.465403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] PATH_INFO: `/v3/projects/d2bb400f81244c0391bf26bbe9ef71e6` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:49.466819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2bb400f81244c0391bf26bbe9ef71e6)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:49.468774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:49.609530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2bb400f81244c0391bf26bbe9ef71e6', 'request_id': 'req-bfb0fae9-4113-469a-83be-5648e493c49e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:49.615903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2bb400f81244c0391bf26bbe9ef71e6', 'request_id': 'req-bfb0fae9-4113-469a-83be-5648e493c49e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:49.621560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2bb400f81244c0391bf26bbe9ef71e6', 'request_id': 'req-bfb0fae9-4113-469a-83be-5648e493c49e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:49.699944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] The token cache is being invalidate because project d2bb400f81244c0391bf26bbe9ef71e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:49.700231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:49.704067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:49.704067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1009/2033] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:49 2022] DELETE /identity/v3/projects/d2bb400f81244c0391bf26bbe9ef71e6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:49.706762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:49.717626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6bca5-efe8-4309-9f49-7a718fbf67a3 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:49.773867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:49.774484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:49.774797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:49.775160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] PATH_INFO: `/v3/users/48289ac2a1384701b8cdbb7c6dac381c` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:49.776641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=48289ac2a1384701b8cdbb7c6dac381c)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:49.779635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:49.987380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48289ac2a1384701b8cdbb7c6dac381c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:49.988042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48289ac2a1384701b8cdbb7c6dac381c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.063024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48289ac2a1384701b8cdbb7c6dac381c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.071766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:50.076998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48289ac2a1384701b8cdbb7c6dac381c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.077680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2e82bf5a-e93f-4ff3-a99a-030bd233b5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48289ac2a1384701b8cdbb7c6dac381c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.090515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1025/2034] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:31:49 2022] DELETE /identity/v3/users/48289ac2a1384701b8cdbb7c6dac381c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:50.531805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:50.534104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:50.547879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0fae9-4113-469a-83be-5648e493c49e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:50.612517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:50.613156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:50.613437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:50.613712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] PATH_INFO: `/v3/projects/f476272449154715a42314be6ef2a169` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:50.615123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f476272449154715a42314be6ef2a169)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:50.617018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:50.740340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f476272449154715a42314be6ef2a169', 'request_id': 'req-9fa0c865-c738-44f1-ae69-96f4d6573b96'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.754889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f476272449154715a42314be6ef2a169', 'request_id': 'req-9fa0c865-c738-44f1-ae69-96f4d6573b96'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.754889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f476272449154715a42314be6ef2a169', 'request_id': 'req-9fa0c865-c738-44f1-ae69-96f4d6573b96'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.828358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] The token cache is being invalidate because project f476272449154715a42314be6ef2a169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:31:50.828644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.829260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-9fa0c865-c738-44f1-ae69-96f4d6573b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:31:50.830879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1010/2035] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:31:50 2022] DELETE /identity/v3/projects/f476272449154715a42314be6ef2a169 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:31:50.835978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70d61-f73d-41f9-88d0-aae3d7c8d4e1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:50.836468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70d61-f73d-41f9-88d0-aae3d7c8d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:50.836557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70d61-f73d-41f9-88d0-aae3d7c8d4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:50.857805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a5e70d61-f73d-41f9-88d0-aae3d7c8d4e1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:50.859086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a5e70d61-f73d-41f9-88d0-aae3d7c8d4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:50.920267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1026/2036] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:50.928990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0106fb0-a6fc-4510-a7bb-7fbf8f512197 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:50.929276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0106fb0-a6fc-4510-a7bb-7fbf8f512197 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:50.929551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0106fb0-a6fc-4510-a7bb-7fbf8f512197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:50.940412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:50.955565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b0106fb0-a6fc-4510-a7bb-7fbf8f512197 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:50.959933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b0106fb0-a6fc-4510-a7bb-7fbf8f512197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:50.969639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1011/2037] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:50.971285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e70d61-f73d-41f9-88d0-aae3d7c8d4e1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:50.985915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e70d61-f73d-41f9-88d0-aae3d7c8d4e1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:50.991107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:50.991633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:50.992000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:50.992143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:50.992787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:31:50.993086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:50.994985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:51.010895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1027/2038] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:31:51.011968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0106fb0-a6fc-4510-a7bb-7fbf8f512197 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:51.022975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0106fb0-a6fc-4510-a7bb-7fbf8f512197 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:51.026707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:51.027853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:51.028121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:51.028387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:51.029019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:51.031006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:51.139371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1012/2039] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:51 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:31:51.140552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:51.150720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcee2f-7975-4ae2-b778-7f984348c22b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:51.208664 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:51.209261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:51.209539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:51.209813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:51.210440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:51.212449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:51.332846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1028/2040] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:51 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:51.336633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:51.347352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a4ec7-c18d-44ca-930b-ccea28bbeb61 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:51.351895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:51.352478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:51.352912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:51.353121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:51.359695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:51.363855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:51.372496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1013/2041] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:51.376712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:51.387570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8729118-7a5a-4ab3-af46-061ef6d1431f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:51.392708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eb71f-a202-4e9b-a28d-ec5e113246e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:51.393406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eb71f-a202-4e9b-a28d-ec5e113246e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:51.393684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eb71f-a202-4e9b-a28d-ec5e113246e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:51.393961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eb71f-a202-4e9b-a28d-ec5e113246e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:51.394600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eb71f-a202-4e9b-a28d-ec5e113246e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:51.398597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eb71f-a202-4e9b-a28d-ec5e113246e5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:51.407760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1029/2042] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:51.411994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:51.425536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a47e6-ca45-44c9-8c99-dddaf2050517 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:51.430161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:51.431644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:51.431644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:51.431644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] PATH_INFO: `/v3/projects/9c09203d6b5d4b99b3431dff1f8c2109/users/8e0492aa392642d9b78aa7811593d717/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:51.446394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c09203d6b5d4b99b3431dff1f8c2109, user_id=8e0492aa392642d9b78aa7811593d717, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:51.449607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:51.549476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1014/2043] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:51 2022] PUT /identity/v3/projects/9c09203d6b5d4b99b3431dff1f8c2109/users/8e0492aa392642d9b78aa7811593d717/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:51.561638 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4099c-bf95-40f7-bad1-c2e5ed049e73 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:51.561848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4099c-bf95-40f7-bad1-c2e5ed049e73 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:51.562367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4099c-bf95-40f7-bad1-c2e5ed049e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:51.597587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c0a4099c-bf95-40f7-bad1-c2e5ed049e73 None None] MFA Rules not processed for user `8e0492aa392642d9b78aa7811593d717`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:51.599071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c0a4099c-bf95-40f7-bad1-c2e5ed049e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:51.646426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:51.681919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1030/2044] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:51.692213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:51.780398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facac29-346d-44ed-8417-504f206225c8 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:51.786706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:51.787723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:51.788066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:51.788432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:51.789166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:51.791256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:51.925249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1015/2045] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:31:51 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:31:51.929292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4099c-bf95-40f7-bad1-c2e5ed049e73 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:51.939919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4099c-bf95-40f7-bad1-c2e5ed049e73 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:51.999808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:52.000409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:52.000686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:52.000957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:52.001613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:52.003608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:52.093923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1031/2046] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:31:51 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:31:52.097899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:52.108225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2d66f-2b7a-4f14-9e48-d9320a132b3b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:52.112788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:52.113368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:52.113642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:52.113912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:52.114837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:52.116500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:52.126740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1016/2047] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:52.130753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:52.141276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012e25-b56e-45f1-a2da-ea2cbb5150b2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:52.145974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:52.146616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:52.146928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:52.147200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] PATH_INFO: `/v3/projects/f331ac0edea8441bae928e47adf93da4/users/32820d927ea5467faa3d1e1300c57d42/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:52.163191 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f331ac0edea8441bae928e47adf93da4, user_id=32820d927ea5467faa3d1e1300c57d42, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:52.165882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:52.231004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1032/2048] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:52 2022] PUT /identity/v3/projects/f331ac0edea8441bae928e47adf93da4/users/32820d927ea5467faa3d1e1300c57d42/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:52.234912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:52.245348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e040f-7dc8-4186-95bc-e0192d9865d9 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:52.304018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601896cd-e00d-47c2-b33c-9c5c985fef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:52.304645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601896cd-e00d-47c2-b33c-9c5c985fef22 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:52.304927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601896cd-e00d-47c2-b33c-9c5c985fef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:52.305199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601896cd-e00d-47c2-b33c-9c5c985fef22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:52.305838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601896cd-e00d-47c2-b33c-9c5c985fef22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:52.307477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601896cd-e00d-47c2-b33c-9c5c985fef22 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:52.318239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1017/2049] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:31:52.324693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:52.336514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696704f-d515-4277-b1d7-16decc787140 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:52.341627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:52.342912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:52.343571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:52.343897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] PATH_INFO: `/v3/projects/f331ac0edea8441bae928e47adf93da4/users/32820d927ea5467faa3d1e1300c57d42/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:52.346741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f331ac0edea8441bae928e47adf93da4, user_id=32820d927ea5467faa3d1e1300c57d42, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:52.349402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:52.361750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1033/2050] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:31:52 2022] PUT /identity/v3/projects/f331ac0edea8441bae928e47adf93da4/users/32820d927ea5467faa3d1e1300c57d42/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:31:52.372252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00f688-b59a-4966-a56c-b7519296d7b6 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:52.372537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00f688-b59a-4966-a56c-b7519296d7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:52.372808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00f688-b59a-4966-a56c-b7519296d7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:52.394845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:52.403247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cf00f688-b59a-4966-a56c-b7519296d7b6 None None] MFA Rules not processed for user `32820d927ea5467faa3d1e1300c57d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:52.405253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cf00f688-b59a-4966-a56c-b7519296d7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:52.475803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1018/2051] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:31:52.487405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:52.489091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:52.548973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e391a-7fa7-4622-b6c9-029162939691 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:52.552784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:52.553381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:52.553703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:52.553970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:52.598927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:52.612218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:52.613645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:52.622534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1034/2052] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:02.499846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=29 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:02.665743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:05.321475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00f688-b59a-4966-a56c-b7519296d7b6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:05.332446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00f688-b59a-4966-a56c-b7519296d7b6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:05.336788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690db04-c7da-43ca-b20d-66b8846796bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:05.337428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690db04-c7da-43ca-b20d-66b8846796bb service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:05.337708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690db04-c7da-43ca-b20d-66b8846796bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:05.337980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690db04-c7da-43ca-b20d-66b8846796bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:05.348765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690db04-c7da-43ca-b20d-66b8846796bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:05.350241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690db04-c7da-43ca-b20d-66b8846796bb service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:05.361771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1019/2053] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:06.535208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:06.536372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:06.622960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a478-e966-43ee-8b70-e0cf50b0a6f0 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:06.627494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:06.628193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:06.628480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:06.628753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] PATH_INFO: `/v3/users/ceba49bce923449ca7a04727965ce796` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:06.629907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceba49bce923449ca7a04727965ce796)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:06.631891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:06.797679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceba49bce923449ca7a04727965ce796'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:06.798345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceba49bce923449ca7a04727965ce796'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:06.851237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceba49bce923449ca7a04727965ce796'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:06.865179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceba49bce923449ca7a04727965ce796'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:06.865828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceba49bce923449ca7a04727965ce796'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:06.879623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1035/2054] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:06 2022] DELETE /identity/v3/users/ceba49bce923449ca7a04727965ce796 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:06.884252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:06.884729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:06.884820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:06.904719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:06.905946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:06.967896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1020/2055] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:06.977944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:06.979359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:07.040102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c32d77-a397-4635-a3f2-3ac6cc5d0ec0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:07.043669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:07.044272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:07.045130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:07.045404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:07.054615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:07.056095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:07.066157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1036/2056] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:07.547753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:07.549806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:07.562448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f022879-5dc2-458f-b91d-41c1d48787d4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:07.567259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:07.567999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:07.568355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:07.568649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] PATH_INFO: `/v3/projects/9189324d1b0f4f2d99fedbb18bde61e9` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:07.570463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9189324d1b0f4f2d99fedbb18bde61e9)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:07.572542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:07.686146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9189324d1b0f4f2d99fedbb18bde61e9', 'request_id': 'req-7d6fdfe0-502e-439c-9a61-e899957cf072'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:07.692266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9189324d1b0f4f2d99fedbb18bde61e9', 'request_id': 'req-7d6fdfe0-502e-439c-9a61-e899957cf072'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:07.697829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9189324d1b0f4f2d99fedbb18bde61e9', 'request_id': 'req-7d6fdfe0-502e-439c-9a61-e899957cf072'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:07.773372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] The token cache is being invalidate because project 9189324d1b0f4f2d99fedbb18bde61e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:07.773661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:07.774281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:07.775803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1021/2057] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:07 2022] DELETE /identity/v3/projects/9189324d1b0f4f2d99fedbb18bde61e9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:07.780642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:07.790396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84777e37-5f4a-4d3d-beb6-f971fd50b607 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:07.808545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:07.847573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:07.848112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:07.848390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:07.848659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] PATH_INFO: `/v3/users/6c3a7b17d53e4d1e9d2275738925a535` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:07.849759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c3a7b17d53e4d1e9d2275738925a535)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:07.851700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:08.057143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c3a7b17d53e4d1e9d2275738925a535'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.057922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c3a7b17d53e4d1e9d2275738925a535'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.127927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c3a7b17d53e4d1e9d2275738925a535'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.140702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c3a7b17d53e4d1e9d2275738925a535'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.141349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e6331e09-74d9-4fb6-a9a5-7218448b6327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c3a7b17d53e4d1e9d2275738925a535'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.154271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1037/2058] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:07 2022] DELETE /identity/v3/users/6c3a7b17d53e4d1e9d2275738925a535 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:08.635943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:08.638180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:08.659065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fdfe0-502e-439c-9a61-e899957cf072 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:08.724551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:08.725200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:08.725479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:08.725748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] PATH_INFO: `/v3/projects/c45727e22bf5496fb71f0dfa2e156d95` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:08.726925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45727e22bf5496fb71f0dfa2e156d95)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:08.728817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:08.741323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:08.838374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45727e22bf5496fb71f0dfa2e156d95', 'request_id': 'req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.844845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45727e22bf5496fb71f0dfa2e156d95', 'request_id': 'req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.850199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45727e22bf5496fb71f0dfa2e156d95', 'request_id': 'req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.943079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] The token cache is being invalidate because project c45727e22bf5496fb71f0dfa2e156d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:08.943285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.943895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7a0f2b6a-e4c4-4ee9-b7d2-0e7cfe772cb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:08.945790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1022/2059] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:08 2022] DELETE /identity/v3/projects/c45727e22bf5496fb71f0dfa2e156d95 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:08.950596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5112c-aab7-419e-b839-f40a2fbfbc3c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:08.951090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5112c-aab7-419e-b839-f40a2fbfbc3c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:08.951798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5112c-aab7-419e-b839-f40a2fbfbc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:08.972859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-52c5112c-aab7-419e-b839-f40a2fbfbc3c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:08.974071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-52c5112c-aab7-419e-b839-f40a2fbfbc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:09.035058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1038/2060] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:09.042895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879da315-a1aa-46ba-8ca8-322a8a524426 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.043178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879da315-a1aa-46ba-8ca8-322a8a524426 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.043473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879da315-a1aa-46ba-8ca8-322a8a524426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.064199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-879da315-a1aa-46ba-8ca8-322a8a524426 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:09.065442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-879da315-a1aa-46ba-8ca8-322a8a524426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:09.077347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1023/2061] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:09.078596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5112c-aab7-419e-b839-f40a2fbfbc3c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:09.088437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5112c-aab7-419e-b839-f40a2fbfbc3c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:09.093370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:09.093969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.094242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.094507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.095178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:09.095473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:09.097067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:09.120012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1039/2062] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:09.125069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879da315-a1aa-46ba-8ca8-322a8a524426 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:09.136080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879da315-a1aa-46ba-8ca8-322a8a524426 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:09.140944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:09.141559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.141844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.142123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.142791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:09.146103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:09.245441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1024/2063] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:09 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:09.250168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:09.260221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44632d64-b39c-4007-a789-03a69d40fc83 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:09.286916 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:09.322353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:09.323107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.323460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.323838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.324510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:09.326574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:09.434728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1040/2064] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:09 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:09.439506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:09.449955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc252af5-0d4c-4cd8-a71a-bbc601f01128 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:09.454913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:09.455635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.455888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.456244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.456995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:09.459210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:09.491744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1025/2065] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:09.495686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:09.512837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65a428-f098-497a-a658-868e4db0fda9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:09.524938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91612eb5-fdc4-4669-9353-530d92680eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:09.525554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91612eb5-fdc4-4669-9353-530d92680eef admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.525832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91612eb5-fdc4-4669-9353-530d92680eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.526101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91612eb5-fdc4-4669-9353-530d92680eef admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.526806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91612eb5-fdc4-4669-9353-530d92680eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:09.528396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91612eb5-fdc4-4669-9353-530d92680eef admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:09.542701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1041/2066] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:09.556453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:09.567347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345f936-8cfd-4f8d-b1a8-34b4a8d06749 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:09.572072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:09.573367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.573650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.573923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] PATH_INFO: `/v3/projects/4194602b59d341a2af0ecf5e6543a43d/users/d01104eaaa2d4ed7ba88e94de43ecac9/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.589095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4194602b59d341a2af0ecf5e6543a43d, user_id=d01104eaaa2d4ed7ba88e94de43ecac9, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:09.592435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:09.652053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1026/2067] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:09 2022] PUT /identity/v3/projects/4194602b59d341a2af0ecf5e6543a43d/users/d01104eaaa2d4ed7ba88e94de43ecac9/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:09.664501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0481b-b9a4-4b39-bad9-a326b009d9cd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.664792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0481b-b9a4-4b39-bad9-a326b009d9cd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.665067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0481b-b9a4-4b39-bad9-a326b009d9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.698687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e2b0481b-b9a4-4b39-bad9-a326b009d9cd None None] MFA Rules not processed for user `d01104eaaa2d4ed7ba88e94de43ecac9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:09.699920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e2b0481b-b9a4-4b39-bad9-a326b009d9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:09.789765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1042/2068] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:09.794179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:09.888937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3fe6e-ad15-4c59-838b-f35d02bdeedf admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:09.896510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:09.897251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:09.897842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:09.898166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:09.898839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:09.905683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:10.067550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1027/2069] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:09 2022] POST /identity/v3/projects => generated 381 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:10.071574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0481b-b9a4-4b39-bad9-a326b009d9cd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:10.083532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0481b-b9a4-4b39-bad9-a326b009d9cd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:10.144092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:10.144743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:10.145020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:10.145434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:10.146085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:10.148077 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:10.243918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1043/2070] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:10 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:10.248520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:10.258889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2a9c6-2e6e-4fc5-b82f-034094751c62 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:10.263471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:10.264210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:10.264487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:10.264756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:10.265397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:10.267241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:10.278696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1028/2071] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:10.283334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:10.293580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2272b0-3427-4bad-86d3-cbec304623b3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:10.299893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:10.300614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:10.300891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:10.301161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] PATH_INFO: `/v3/projects/a32b589849da406dbe5c16e31ee5e514/users/d6f932909c604cd391892babb03c2bd4/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:10.317696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a32b589849da406dbe5c16e31ee5e514, user_id=d6f932909c604cd391892babb03c2bd4, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:10.320648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:10.393524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1044/2072] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:10 2022] PUT /identity/v3/projects/a32b589849da406dbe5c16e31ee5e514/users/d6f932909c604cd391892babb03c2bd4/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:10.398052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:10.421870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca6db9-2b7f-41ed-9f02-0864b1bb1e88 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:10.438418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:10.505432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3cbe7-5c03-49fb-ab5b-174b26075a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:10.506070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3cbe7-5c03-49fb-ab5b-174b26075a62 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:10.506334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3cbe7-5c03-49fb-ab5b-174b26075a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:10.506592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3cbe7-5c03-49fb-ab5b-174b26075a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:10.507287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3cbe7-5c03-49fb-ab5b-174b26075a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:10.508888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3cbe7-5c03-49fb-ab5b-174b26075a62 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:10.521859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1029/2073] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:10.526376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:10.534616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:10.539091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f03ad-718d-433a-b8e6-590230819448 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:10.546737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:10.547745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:10.548127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:10.548685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] PATH_INFO: `/v3/projects/a32b589849da406dbe5c16e31ee5e514/users/d6f932909c604cd391892babb03c2bd4/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:10.551056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a32b589849da406dbe5c16e31ee5e514, user_id=d6f932909c604cd391892babb03c2bd4, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:10.554362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:10.568373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1045/2074] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:10 2022] PUT /identity/v3/projects/a32b589849da406dbe5c16e31ee5e514/users/d6f932909c604cd391892babb03c2bd4/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:10.580699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77e7ca-5c26-4acd-803f-8876c7d4a180 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:10.580974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77e7ca-5c26-4acd-803f-8876c7d4a180 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:10.581248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77e7ca-5c26-4acd-803f-8876c7d4a180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:10.617436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9c77e7ca-5c26-4acd-803f-8876c7d4a180 None None] MFA Rules not processed for user `d6f932909c604cd391892babb03c2bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:10.618952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9c77e7ca-5c26-4acd-803f-8876c7d4a180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:10.698773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1030/2075] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:10.710992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:10.712985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:10.811136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d82c55-aa65-4121-a45f-5a3e1ce1132a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:10.818394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:10.818394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:10.818394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:10.818394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:10.831430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:10.832981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:10.850451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1046/2076] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:11.083498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77e7ca-5c26-4acd-803f-8876c7d4a180 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:11.094119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77e7ca-5c26-4acd-803f-8876c7d4a180 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:11.097984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611d699-30d9-48ad-af35-9314531a63ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:11.098762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611d699-30d9-48ad-af35-9314531a63ce service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:11.099168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611d699-30d9-48ad-af35-9314531a63ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:11.099544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611d699-30d9-48ad-af35-9314531a63ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:11.181855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611d699-30d9-48ad-af35-9314531a63ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:11.184369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611d699-30d9-48ad-af35-9314531a63ce service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:11.199478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1031/2077] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:12.540224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:12.541647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:12.618173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f39a-9b2d-4422-9555-6fcf72930b07 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:12.623756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:12.624329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:12.624606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:12.624878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] PATH_INFO: `/v3/users/8e0492aa392642d9b78aa7811593d717` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:12.626067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0492aa392642d9b78aa7811593d717)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:12.628039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:13.078740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0492aa392642d9b78aa7811593d717'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:13.079850 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0492aa392642d9b78aa7811593d717'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:13.134520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0492aa392642d9b78aa7811593d717'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:13.150523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0492aa392642d9b78aa7811593d717'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:13.151328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0492aa392642d9b78aa7811593d717'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:13.166977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1047/2078] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:12 2022] DELETE /identity/v3/users/8e0492aa392642d9b78aa7811593d717 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:13.172219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f76bf-1940-4b2b-81de-72662733a6e1 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:13.172219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f76bf-1940-4b2b-81de-72662733a6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:13.172219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f76bf-1940-4b2b-81de-72662733a6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:13.211543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-045f76bf-1940-4b2b-81de-72662733a6e1 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:13.214148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-045f76bf-1940-4b2b-81de-72662733a6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:13.273476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:13.333122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1032/2079] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:13.341285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:13.342743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:13.356232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:13.404697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f994b8-6933-4167-9b59-b17782f8c0bc admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:13.408305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67eb3a2-0ca0-4e42-aba0-38585e545294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:13.408909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eb3a2-0ca0-4e42-aba0-38585e545294 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:13.409188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eb3a2-0ca0-4e42-aba0-38585e545294 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:13.409450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eb3a2-0ca0-4e42-aba0-38585e545294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:13.421739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67eb3a2-0ca0-4e42-aba0-38585e545294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:13.426181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67eb3a2-0ca0-4e42-aba0-38585e545294 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:13.437291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1048/2080] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:13.696208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53d70d-79d9-4a86-8c0b-fb0098382ee6 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:13.696494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53d70d-79d9-4a86-8c0b-fb0098382ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:13.696950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53d70d-79d9-4a86-8c0b-fb0098382ee6 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:13.698500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1033/2081] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:13.711067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:13.711960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:13.712402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:13.742478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:13.744121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:13.826451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1049/2082] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:13.837448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53d70d-79d9-4a86-8c0b-fb0098382ee6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:13.839246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fc53d70d-79d9-4a86-8c0b-fb0098382ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:13.862260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53d70d-79d9-4a86-8c0b-fb0098382ee6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:13.884617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:13.885243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:13.885690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:13.886081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:13.899106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:13.904894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:13.905246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:13.905823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:13.917084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1539284-6835-4681-ad7b-1f807e40d62f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:13.917247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1034/2083] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:13.928724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:13.929413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:13.929645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:13.935033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] PATH_INFO: `/v3/users/281e6cd581c54901b8feaa291db588eb` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:13.937422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=281e6cd581c54901b8feaa291db588eb)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:13.939883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:14.066406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:14.067847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:14.081382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60759eb-bf7b-4486-b324-c12d41edb9a5 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:14.086193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:14.087013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:14.087378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:14.087679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] PATH_INFO: `/v3/projects/9c09203d6b5d4b99b3431dff1f8c2109` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:14.089507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c09203d6b5d4b99b3431dff1f8c2109)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:14.092733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:14.150033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281e6cd581c54901b8feaa291db588eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.150715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281e6cd581c54901b8feaa291db588eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.206586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c09203d6b5d4b99b3431dff1f8c2109', 'request_id': 'req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.213203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c09203d6b5d4b99b3431dff1f8c2109', 'request_id': 'req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.213801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281e6cd581c54901b8feaa291db588eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.221109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c09203d6b5d4b99b3431dff1f8c2109', 'request_id': 'req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.227719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281e6cd581c54901b8feaa291db588eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.228490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281e6cd581c54901b8feaa291db588eb'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.241580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1050/2084] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:13 2022] DELETE /identity/v3/users/281e6cd581c54901b8feaa291db588eb => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:14.349415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] The token cache is being invalidate because project 9c09203d6b5d4b99b3431dff1f8c2109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:14.349415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.349415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.349415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1035/2085] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:14 2022] DELETE /identity/v3/projects/9c09203d6b5d4b99b3431dff1f8c2109 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:14.353158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:14.354581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:14.382808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:14.424537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbb84c-0579-42ed-a1f2-669220c90286 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:14.432839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:14.432839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:14.432839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:14.432839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] PATH_INFO: `/v3/users/32820d927ea5467faa3d1e1300c57d42` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:14.433236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32820d927ea5467faa3d1e1300c57d42)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:14.433752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:14.554892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:14.554892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:14.627107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44dfa-5d4d-4704-93a7-4765b7cf1a45 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:14.632098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:14.632734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:14.633017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:14.633290 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] PATH_INFO: `/v3/projects/43ef4ba417014205b91496a8037540ed` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:14.634474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ef4ba417014205b91496a8037540ed)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:14.636460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:14.697517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32820d927ea5467faa3d1e1300c57d42'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.698173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32820d927ea5467faa3d1e1300c57d42'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.740525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:14.744226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ef4ba417014205b91496a8037540ed', 'request_id': 'req-88e5b5c6-8e44-49b3-a7f4-747e1299981c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.751155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ef4ba417014205b91496a8037540ed', 'request_id': 'req-88e5b5c6-8e44-49b3-a7f4-747e1299981c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.756766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ef4ba417014205b91496a8037540ed', 'request_id': 'req-88e5b5c6-8e44-49b3-a7f4-747e1299981c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.782519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32820d927ea5467faa3d1e1300c57d42'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.795053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32820d927ea5467faa3d1e1300c57d42'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.795581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24ae0ac2-2dda-4edf-8490-eca2a2a9f158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32820d927ea5467faa3d1e1300c57d42'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.813634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1051/2086] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:14 2022] DELETE /identity/v3/users/32820d927ea5467faa3d1e1300c57d42 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:14.864811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] The token cache is being invalidate because project 43ef4ba417014205b91496a8037540ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:14.865098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.865732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-88e5b5c6-8e44-49b3-a7f4-747e1299981c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:14.867894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1036/2087] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:14 2022] DELETE /identity/v3/projects/43ef4ba417014205b91496a8037540ed => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:14.875699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf6a9b-1b0c-481e-8870-583b8eba9e65 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:14.876484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf6a9b-1b0c-481e-8870-583b8eba9e65 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:14.877062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf6a9b-1b0c-481e-8870-583b8eba9e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:14.901524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-37cf6a9b-1b0c-481e-8870-583b8eba9e65 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:14.903684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-37cf6a9b-1b0c-481e-8870-583b8eba9e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:14.977423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1052/2088] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:14.988026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecae2b-d6f7-46cb-a6aa-8f24806f13e7 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:14.988326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecae2b-d6f7-46cb-a6aa-8f24806f13e7 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:14.988603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecae2b-d6f7-46cb-a6aa-8f24806f13e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.017591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-17ecae2b-d6f7-46cb-a6aa-8f24806f13e7 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:15.020208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-17ecae2b-d6f7-46cb-a6aa-8f24806f13e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:15.028883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1037/2089] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:15.033195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf6a9b-1b0c-481e-8870-583b8eba9e65 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.043110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf6a9b-1b0c-481e-8870-583b8eba9e65 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.049031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.049031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.049031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.049348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.049963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:15.050267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.052381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.066265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1053/2090] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:15.082376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecae2b-d6f7-46cb-a6aa-8f24806f13e7 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.092487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecae2b-d6f7-46cb-a6aa-8f24806f13e7 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.099895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.100870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.100870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.103361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.103361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.104565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.215887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1038/2091] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:15 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:15.217659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.227939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2915651-3f61-4ebc-9073-be70dc90abc6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.255349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:15.286007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.286683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.287110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.287220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.287835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.289725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.295872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.297847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:15.312757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d0a2f-1489-416b-abd4-1353750f72e7 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.317746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.318402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.318720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.319003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] PATH_INFO: `/v3/projects/f331ac0edea8441bae928e47adf93da4` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.320138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f331ac0edea8441bae928e47adf93da4)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.322052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.392118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1054/2092] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:15 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:15.395954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.406097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0b3d9-33c8-43e6-aec4-b2986a8ddfee admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.410611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.411207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.411488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.411933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.412806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.418644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.428401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1055/2093] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:15.437751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.448180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ce7ed-7a0d-4c67-aa69-3d3e12de12c9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.453825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.457994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.457994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.457994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.457994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.457994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.468360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1056/2094] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:15.472448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.481904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae56c1-c8a5-4856-8f23-1ca83a928367 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.489581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19d521-f06d-4bf9-ab0a-11b0396e2f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.490244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19d521-f06d-4bf9-ab0a-11b0396e2f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.490523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19d521-f06d-4bf9-ab0a-11b0396e2f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.490842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19d521-f06d-4bf9-ab0a-11b0396e2f5b admin admin] PATH_INFO: `/v3/projects/160c47318b954399b4b19a977a16759b/users/dbb6e9196121494e8eb1b56f1b2cce31/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.501355 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f331ac0edea8441bae928e47adf93da4', 'request_id': 'req-335eb837-ee4c-4cd6-8614-d0186895f707'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:15.507529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f331ac0edea8441bae928e47adf93da4', 'request_id': 'req-335eb837-ee4c-4cd6-8614-d0186895f707'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:15.507753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19d521-f06d-4bf9-ab0a-11b0396e2f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=160c47318b954399b4b19a977a16759b, user_id=dbb6e9196121494e8eb1b56f1b2cce31, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.510438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19d521-f06d-4bf9-ab0a-11b0396e2f5b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.514149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f331ac0edea8441bae928e47adf93da4', 'request_id': 'req-335eb837-ee4c-4cd6-8614-d0186895f707'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:15.591856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1057/2095] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:15 2022] PUT /identity/v3/projects/160c47318b954399b4b19a977a16759b/users/dbb6e9196121494e8eb1b56f1b2cce31/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:15.601936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] The token cache is being invalidate because project f331ac0edea8441bae928e47adf93da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:15.602112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.602215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:15.602332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.602605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.602891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-335eb837-ee4c-4cd6-8614-d0186895f707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:15.604548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1039/2096] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:15 2022] DELETE /identity/v3/projects/f331ac0edea8441bae928e47adf93da4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:15.610052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf183d-0940-4513-83ce-d507aed8c377 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.610338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf183d-0940-4513-83ce-d507aed8c377 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.610607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf183d-0940-4513-83ce-d507aed8c377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.632481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] MFA Rules not processed for user `dbb6e9196121494e8eb1b56f1b2cce31`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:15.633705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:15.635180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c1bf183d-0940-4513-83ce-d507aed8c377 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:15.636359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c1bf183d-0940-4513-83ce-d507aed8c377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:15.696643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1040/2097] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:15.705365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30538e-97fb-42b9-9e11-f719e378445b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.705647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30538e-97fb-42b9-9e11-f719e378445b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.705918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30538e-97fb-42b9-9e11-f719e378445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.712434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1058/2098] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:15.720329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.721869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:15.725030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2e30538e-97fb-42b9-9e11-f719e378445b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:15.726195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2e30538e-97fb-42b9-9e11-f719e378445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:15.734637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1041/2099] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:15.738935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30538e-97fb-42b9-9e11-f719e378445b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.750594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30538e-97fb-42b9-9e11-f719e378445b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.755665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.756854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.757134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.757400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.758050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:15.758487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.760123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.777330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1042/2100] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:32:15.779407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.790591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d843f78-89b6-4d9f-a83b-62d73318ccf5 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.798334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:15.799558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.800289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.800618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.800985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.801918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.804032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.812195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:15.816849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83741c7a-239b-490b-b6d9-5612e31efe8f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:15.823553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:15.825157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:15.825157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:15.825157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:15.833956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:15.838324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:15.845312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1059/2101] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:15.946543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1043/2102] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:15 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:15.951017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:15.964353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a68e-b3ce-4f6f-8e74-f57147394692 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:16.034463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:16.035095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:16.035371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:16.035644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:16.036288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:16.038166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:16.118162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1060/2103] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:15 2022] POST /identity/v3/users => generated 281 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:16.122398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:16.133628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea45c1-1c33-4397-9ef6-796c373858ce admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:16.138493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:16.139199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:16.139547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:16.139890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:16.140609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:16.142390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:16.153178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1044/2104] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:16.157199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:16.167327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94c8c3-d6b3-45be-96ea-c02ae9b022a2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:16.172497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fea6f-1731-4051-b79f-365c8a338e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:16.173157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fea6f-1731-4051-b79f-365c8a338e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:16.173499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fea6f-1731-4051-b79f-365c8a338e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:16.173833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fea6f-1731-4051-b79f-365c8a338e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:16.174534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fea6f-1731-4051-b79f-365c8a338e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:16.176325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fea6f-1731-4051-b79f-365c8a338e91 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:16.188313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1061/2105] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:16.192176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:16.203826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfa1c6-31fb-43f7-8e3a-d294af742483 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:16.207665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:16.208319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:16.208602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:16.208889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] PATH_INFO: `/v3/projects/c0113f94888243c78140bb01be221fec/users/2f3a0ce4b2d3453a98f1baaad0da9fa0/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:16.224164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0113f94888243c78140bb01be221fec, user_id=2f3a0ce4b2d3453a98f1baaad0da9fa0, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:16.226946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:16.299287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1045/2106] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:16 2022] PUT /identity/v3/projects/c0113f94888243c78140bb01be221fec/users/2f3a0ce4b2d3453a98f1baaad0da9fa0/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:16.310877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bff900-55b0-4555-8167-e6ad6d7d6de8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:16.311161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bff900-55b0-4555-8167-e6ad6d7d6de8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:16.311432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bff900-55b0-4555-8167-e6ad6d7d6de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:16.339132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d4bff900-55b0-4555-8167-e6ad6d7d6de8 None None] MFA Rules not processed for user `2f3a0ce4b2d3453a98f1baaad0da9fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:16.340328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d4bff900-55b0-4555-8167-e6ad6d7d6de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:16.412846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1062/2107] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:16.526327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:16.529934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:16.585582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f046cb-ebbf-436b-867f-6caca1da30eb admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:16.589337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b244206-cebd-4802-9477-f016420e5960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:16.589933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244206-cebd-4802-9477-f016420e5960 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:16.590213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244206-cebd-4802-9477-f016420e5960 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:16.590486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244206-cebd-4802-9477-f016420e5960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:16.599309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b244206-cebd-4802-9477-f016420e5960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:16.600796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b244206-cebd-4802-9477-f016420e5960 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:16.610285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1046/2108] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:16.626613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bff900-55b0-4555-8167-e6ad6d7d6de8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:16.687490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:16.694202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bff900-55b0-4555-8167-e6ad6d7d6de8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:16.699050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:16.699668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:16.699942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:16.700209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] PATH_INFO: `/v3/users/2f3a0ce4b2d3453a98f1baaad0da9fa0` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:16.701845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f3a0ce4b2d3453a98f1baaad0da9fa0)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:16.703816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:16.903808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f3a0ce4b2d3453a98f1baaad0da9fa0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:16.903808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f3a0ce4b2d3453a98f1baaad0da9fa0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:16.978698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f3a0ce4b2d3453a98f1baaad0da9fa0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:16.991278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f3a0ce4b2d3453a98f1baaad0da9fa0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:16.991924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f3a0ce4b2d3453a98f1baaad0da9fa0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:17.004738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1063/2109] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:16 2022] DELETE /identity/v3/users/2f3a0ce4b2d3453a98f1baaad0da9fa0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:17.009272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:17.009563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:17.009841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:17.029108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:17.030309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:17.088288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1047/2110] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:17.098146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:17.099558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:17.156463 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0671c80-12a4-4bc3-8b2a-36345869dc38 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:17.160038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a3678-9b78-4029-9311-09df0bc25296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:17.160624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a3678-9b78-4029-9311-09df0bc25296 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:17.160903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a3678-9b78-4029-9311-09df0bc25296 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:17.161170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a3678-9b78-4029-9311-09df0bc25296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:17.169926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a3678-9b78-4029-9311-09df0bc25296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:17.171373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a3678-9b78-4029-9311-09df0bc25296 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:17.180611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1064/2111] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:17.603529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:17.605566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:17.617850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa49c0-9e9c-4c84-8867-66bc369b25b6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:17.623297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:17.624044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:17.624318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:17.624591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] PATH_INFO: `/v3/projects/c0113f94888243c78140bb01be221fec` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:17.625783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0113f94888243c78140bb01be221fec)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:17.627974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:17.730220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:17.731411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0113f94888243c78140bb01be221fec', 'request_id': 'req-25ac5610-b697-450e-ba11-1b613736a296'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:17.738027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0113f94888243c78140bb01be221fec', 'request_id': 'req-25ac5610-b697-450e-ba11-1b613736a296'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:17.744118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0113f94888243c78140bb01be221fec', 'request_id': 'req-25ac5610-b697-450e-ba11-1b613736a296'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:17.813151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] The token cache is being invalidate because project c0113f94888243c78140bb01be221fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:17.813465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:17.817637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-25ac5610-b697-450e-ba11-1b613736a296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:17.817637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1048/2112] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:17 2022] DELETE /identity/v3/projects/c0113f94888243c78140bb01be221fec => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:17.821430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31894846-0fef-4959-ae5f-172e742adb6e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:17.821743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31894846-0fef-4959-ae5f-172e742adb6e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:17.822018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31894846-0fef-4959-ae5f-172e742adb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:17.841893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-31894846-0fef-4959-ae5f-172e742adb6e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:17.843124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-31894846-0fef-4959-ae5f-172e742adb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:17.901237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1065/2113] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:17.910512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96526c7f-f2f3-4c92-aa45-f357f847d551 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:17.910826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96526c7f-f2f3-4c92-aa45-f357f847d551 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:17.911095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96526c7f-f2f3-4c92-aa45-f357f847d551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:17.930847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-96526c7f-f2f3-4c92-aa45-f357f847d551 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:17.932033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-96526c7f-f2f3-4c92-aa45-f357f847d551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:17.939829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1049/2114] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:17.944048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31894846-0fef-4959-ae5f-172e742adb6e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:17.953696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31894846-0fef-4959-ae5f-172e742adb6e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:17.958382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:17.959002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:17.959281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:17.959567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:17.960330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:17.960631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:17.962219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:17.974300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1066/2115] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:17.978493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96526c7f-f2f3-4c92-aa45-f357f847d551 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:17.988189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96526c7f-f2f3-4c92-aa45-f357f847d551 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:17.993050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:17.993635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:17.993910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:17.994179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:17.994836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:17.996715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:18.075181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1050/2116] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:17 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:18.079460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:18.085870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:18.089481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89290888-4a22-493b-89e8-1bc31e565f8a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:18.144908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:18.145502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.146126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.146445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:18.147110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:18.149047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:18.270809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1067/2117] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:18 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:18.274629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:18.285610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07c1d4-d941-4b71-960c-dc3d3ca1a35f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:18.290141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:18.290752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.291031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.291304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:18.291937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:18.293525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:18.303383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1051/2118] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:18.307306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:18.317190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b042e-0afa-409d-bd56-4a94f642d219 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:18.321670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae358c-8a91-438d-9b37-59574495b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:18.322250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae358c-8a91-438d-9b37-59574495b565 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.322528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae358c-8a91-438d-9b37-59574495b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.322840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae358c-8a91-438d-9b37-59574495b565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:18.323466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae358c-8a91-438d-9b37-59574495b565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:18.325035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae358c-8a91-438d-9b37-59574495b565 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:18.335786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1068/2119] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:18.339958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:18.350256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d016018-746f-4542-ab39-5352c88e2618 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:18.354849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:18.355478 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.355755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.356025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] PATH_INFO: `/v3/projects/f131aa47c61e4c338885741c49918e4c/users/a0a02bd98d434676a6e0e30ed07f8f8c/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:18.370236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f131aa47c61e4c338885741c49918e4c, user_id=a0a02bd98d434676a6e0e30ed07f8f8c, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:18.373020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:18.439794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1052/2120] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:18 2022] PUT /identity/v3/projects/f131aa47c61e4c338885741c49918e4c/users/a0a02bd98d434676a6e0e30ed07f8f8c/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:18.449912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.450192 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.450526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:18.481234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] MFA Rules not processed for user `a0a02bd98d434676a6e0e30ed07f8f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:18.482451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:18.577280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1069/2121] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:18.584679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:18.586433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:18.643349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410063f1-56b2-4de8-b364-904940c6c771 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:18.646925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cb4a0-5065-4948-be4f-7e39914de92c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f33e1c573c044f086f99eb128c8f726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:18.647520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cb4a0-5065-4948-be4f-7e39914de92c service glance] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.647795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cb4a0-5065-4948-be4f-7e39914de92c service glance] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.648066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cb4a0-5065-4948-be4f-7e39914de92c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:18.656300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cb4a0-5065-4948-be4f-7e39914de92c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:18.657717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cb4a0-5065-4948-be4f-7e39914de92c service glance] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:18.666334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1053/2122] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:28.565180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:28.662426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:28.758198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:28.759211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:28.845204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c948e-d85c-4062-84fb-f3a9633fb08e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:28.849881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:28.850510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:28.850817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:28.851092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] PATH_INFO: `/v3/users/dbb6e9196121494e8eb1b56f1b2cce31` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:28.852211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb6e9196121494e8eb1b56f1b2cce31)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:28.854114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:29.005495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb6e9196121494e8eb1b56f1b2cce31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.006180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb6e9196121494e8eb1b56f1b2cce31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.077367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb6e9196121494e8eb1b56f1b2cce31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.100112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb6e9196121494e8eb1b56f1b2cce31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.100961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb6e9196121494e8eb1b56f1b2cce31'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.116856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1070/2123] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:28 2022] DELETE /identity/v3/users/dbb6e9196121494e8eb1b56f1b2cce31 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:29.124475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:29.125657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:29.126468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:29.152447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:29.153977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:29.212014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1054/2124] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:29.221678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:29.223087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:29.281661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60ae1e-55f3-4a96-8bce-7a9b38d51205 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:29.285238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a192d-1c20-48b3-b8e9-893fefcb9e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:29.285839 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a192d-1c20-48b3-b8e9-893fefcb9e4b service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:29.286118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a192d-1c20-48b3-b8e9-893fefcb9e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:29.286391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a192d-1c20-48b3-b8e9-893fefcb9e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:29.296098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a192d-1c20-48b3-b8e9-893fefcb9e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:29.297529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a192d-1c20-48b3-b8e9-893fefcb9e4b service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:29.307144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1071/2125] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:29.749590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:29.751348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:29.763770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01698e11-5995-4d0b-9539-f13abd307d2e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:29.769229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:29.769873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:29.770155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:29.770431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] PATH_INFO: `/v3/projects/160c47318b954399b4b19a977a16759b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:29.771691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=160c47318b954399b4b19a977a16759b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:29.773704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:29.914412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160c47318b954399b4b19a977a16759b', 'request_id': 'req-47131571-5b09-4439-9474-7973196ed15b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.921328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160c47318b954399b4b19a977a16759b', 'request_id': 'req-47131571-5b09-4439-9474-7973196ed15b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.927074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160c47318b954399b4b19a977a16759b', 'request_id': 'req-47131571-5b09-4439-9474-7973196ed15b'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.022364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] The token cache is being invalidate because project 160c47318b954399b4b19a977a16759b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:30.022675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.023334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47131571-5b09-4439-9474-7973196ed15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.024838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1055/2126] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:29 2022] DELETE /identity/v3/projects/160c47318b954399b4b19a977a16759b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:30.030217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91702d1-9de4-4611-b886-db07455396c6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.030737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91702d1-9de4-4611-b886-db07455396c6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.030830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91702d1-9de4-4611-b886-db07455396c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.051090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a91702d1-9de4-4611-b886-db07455396c6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:30.052321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a91702d1-9de4-4611-b886-db07455396c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.069658 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:30.115079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1072/2127] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:30.122470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91399f-65d4-402f-8a70-693799b2cba5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.122789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91399f-65d4-402f-8a70-693799b2cba5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.123066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91399f-65d4-402f-8a70-693799b2cba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.143806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-6c91399f-65d4-402f-8a70-693799b2cba5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:30.145011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6c91399f-65d4-402f-8a70-693799b2cba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.164019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1056/2128] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:30.168215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91702d1-9de4-4611-b886-db07455396c6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.178972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91702d1-9de4-4611-b886-db07455396c6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.190503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.191081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.191362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.192726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.194816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:30.195423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.197038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.219859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1073/2129] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:30.220329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91399f-65d4-402f-8a70-693799b2cba5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.232104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91399f-65d4-402f-8a70-693799b2cba5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.255258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.255882 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.256241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.256505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.257134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.259121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.350328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1057/2130] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:30.352079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.363121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a05a0-bfab-4a68-8abf-0dbe61d89fc7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.421253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.421886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.422162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.422436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.423095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.425046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.482328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.484114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.496836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1fc7d-909f-4821-af49-361621f0a54a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.501893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.502500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.505927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.505927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] PATH_INFO: `/v3/users/d01104eaaa2d4ed7ba88e94de43ecac9` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.505927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01104eaaa2d4ed7ba88e94de43ecac9)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.506202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.535150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1074/2131] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:30.536455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.546232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9bff-9dbf-4159-bb05-6a7d702a5cd3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.550912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.551476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.551750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.552016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.552713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.554276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.565950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1075/2132] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:30.569394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.579290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b499a-b64a-4f7c-b74e-9d63b3e686c2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.583893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.584437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.584708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.584974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.585570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.587190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.597081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1076/2133] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:30.600849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.610029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a67e2a-057f-4b16-a6f0-baf20b03042a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.614923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2fc76-9ad7-44dc-99d5-c464d21e97fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.615532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2fc76-9ad7-44dc-99d5-c464d21e97fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.615812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2fc76-9ad7-44dc-99d5-c464d21e97fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.616084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2fc76-9ad7-44dc-99d5-c464d21e97fb admin admin] PATH_INFO: `/v3/projects/1f7f15ba0b694082a3bdc09ce3ca5c56/users/1e0fe88d979b40b8a570eb7532b21b68/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.630673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2fc76-9ad7-44dc-99d5-c464d21e97fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7f15ba0b694082a3bdc09ce3ca5c56, user_id=1e0fe88d979b40b8a570eb7532b21b68, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.633297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2fc76-9ad7-44dc-99d5-c464d21e97fb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.729187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01104eaaa2d4ed7ba88e94de43ecac9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.729843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01104eaaa2d4ed7ba88e94de43ecac9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.730453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01104eaaa2d4ed7ba88e94de43ecac9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.732448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:30.737529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01104eaaa2d4ed7ba88e94de43ecac9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.745255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1077/2134] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:30 2022] PUT /identity/v3/projects/1f7f15ba0b694082a3bdc09ce3ca5c56/users/1e0fe88d979b40b8a570eb7532b21b68/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:30.755044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ee23d14-abd6-4b03-8c24-e4d36dcd7d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01104eaaa2d4ed7ba88e94de43ecac9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.756303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.756586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.756859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.785384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] MFA Rules not processed for user `1e0fe88d979b40b8a570eb7532b21b68`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:30.786768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.829165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:30.844848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1058/2135] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:30 2022] DELETE /identity/v3/users/d01104eaaa2d4ed7ba88e94de43ecac9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:30.849435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.849727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.850076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.851778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1078/2136] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:30.856055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.858085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.871328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:30.873323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.933613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb533f-3f4d-45f2-8160-41d4f6040670 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.955937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1059/2137] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:30.965268 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.966806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.994103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.994731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.995193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.995339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.995881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.998407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.026769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510825-9f90-4afc-8e07-0f380fcce404 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.030468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.031146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.031421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.031689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.040381 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.041796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.051610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1060/2138] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:31.107056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1079/2139] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:31.110991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.123082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9520032-348b-4be2-9a4d-c6ece40de168 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.179788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.180411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.181069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.181341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.181949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.184012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.289620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:31.298498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1061/2140] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:31.302621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.317278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07e108-08b4-4d2b-8a95-0c9b02b869f3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.323135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.324419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.325134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.325685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.326679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.328511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.342880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1080/2141] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:31.345470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.358514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189e245-392f-4e5f-8f67-96720d5a6cc1 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.363513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.364167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.364732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.365007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] PATH_INFO: `/v3/projects/fee22f0cc4e84bd9a33ec15e89ef91de/users/fcfd3feeca3d4c0e97816eb6789a54fe/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.382690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee22f0cc4e84bd9a33ec15e89ef91de, user_id=fcfd3feeca3d4c0e97816eb6789a54fe, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.390311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.468755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1062/2142] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:31 2022] PUT /identity/v3/projects/fee22f0cc4e84bd9a33ec15e89ef91de/users/fcfd3feeca3d4c0e97816eb6789a54fe/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:31.473504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.483742 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec47ed3-56f6-460b-9d17-6383451ba3f2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.549565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.550200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.550495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.550784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.551425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.553055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.559290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:31.565439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1081/2143] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:31.569675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.580889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.581029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a1f4a-aa18-448d-9f56-53b7b36ac227 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.583198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.586459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c0174-eaea-4f08-bb52-1560ed7298aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.589375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0174-eaea-4f08-bb52-1560ed7298aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.589375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0174-eaea-4f08-bb52-1560ed7298aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.589375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0174-eaea-4f08-bb52-1560ed7298aa admin admin] PATH_INFO: `/v3/projects/fee22f0cc4e84bd9a33ec15e89ef91de/users/fcfd3feeca3d4c0e97816eb6789a54fe/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.592329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c0174-eaea-4f08-bb52-1560ed7298aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee22f0cc4e84bd9a33ec15e89ef91de, user_id=fcfd3feeca3d4c0e97816eb6789a54fe, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.595202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c0174-eaea-4f08-bb52-1560ed7298aa admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.609087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1063/2144] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:31 2022] PUT /identity/v3/projects/fee22f0cc4e84bd9a33ec15e89ef91de/users/fcfd3feeca3d4c0e97816eb6789a54fe/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:31.611415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217a4da-0e6d-451d-9c73-67497d664bdb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.623189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.628800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.629154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.663813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] MFA Rules not processed for user `fcfd3feeca3d4c0e97816eb6789a54fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:31.665181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.692851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.693478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.693754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.694023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] PATH_INFO: `/v3/projects/4194602b59d341a2af0ecf5e6543a43d` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.695207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4194602b59d341a2af0ecf5e6543a43d)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.697089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.737695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1064/2145] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:31.755232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.756923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.815379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4194602b59d341a2af0ecf5e6543a43d', 'request_id': 'req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.818006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f8873-2370-4f20-b0af-7ef209076074 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.821101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4194602b59d341a2af0ecf5e6543a43d', 'request_id': 'req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.827960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4194602b59d341a2af0ecf5e6543a43d', 'request_id': 'req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.874148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.874744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.875024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.875293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.892325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:31.901081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] The token cache is being invalidate because project 4194602b59d341a2af0ecf5e6543a43d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:31.901369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.902041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.903643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1082/2146] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:31 2022] DELETE /identity/v3/projects/4194602b59d341a2af0ecf5e6543a43d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:31.908225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.909909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.938739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.940173 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.949976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1065/2147] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:31.991085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f2c03-f4f8-42ef-8cd5-40bf7098dba3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.004471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.007422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.007422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.007422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] PATH_INFO: `/v3/users/d6f932909c604cd391892babb03c2bd4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.009395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f932909c604cd391892babb03c2bd4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.015042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.230026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:32.243061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f932909c604cd391892babb03c2bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:32.243975 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f932909c604cd391892babb03c2bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:32.322253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f932909c604cd391892babb03c2bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:32.341483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f932909c604cd391892babb03c2bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:32.342227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f932909c604cd391892babb03c2bd4'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:32.354879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1083/2148] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:31 2022] DELETE /identity/v3/users/d6f932909c604cd391892babb03c2bd4 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:32.855793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.868099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4635-92eb-4d49-ac52-39c1f9fc1660 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.931617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.932275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.932553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.932821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] PATH_INFO: `/v3/projects/a32b589849da406dbe5c16e31ee5e514` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.933966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a32b589849da406dbe5c16e31ee5e514)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.939291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.061954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a32b589849da406dbe5c16e31ee5e514', 'request_id': 'req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.061954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a32b589849da406dbe5c16e31ee5e514', 'request_id': 'req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.063301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.064365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a32b589849da406dbe5c16e31ee5e514', 'request_id': 'req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.064820 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:33.132522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f44b-b783-4c79-b647-383a3e1a530b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.148325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] The token cache is being invalidate because project a32b589849da406dbe5c16e31ee5e514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:33.148675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.149327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c07c472-ea1a-4b59-8d88-7e9111fbf0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.151067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1066/2149] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:32 2022] DELETE /identity/v3/projects/a32b589849da406dbe5c16e31ee5e514 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:33.155712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42d5ae-c8fd-4a73-96af-12b542979040 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.155997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42d5ae-c8fd-4a73-96af-12b542979040 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.156272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42d5ae-c8fd-4a73-96af-12b542979040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.177494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-af42d5ae-c8fd-4a73-96af-12b542979040 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:33.178880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-af42d5ae-c8fd-4a73-96af-12b542979040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:33.199528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfd3feeca3d4c0e97816eb6789a54fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fee22f0cc4e84bd9a33ec15e89ef91de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.200278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.200569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.200842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.201563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:33.201794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.203503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.219120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1084/2150] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:32:33.223062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.225141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:33.225747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:33.238725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dab45-f253-460e-9f75-6e194231f331 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.244103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfd3feeca3d4c0e97816eb6789a54fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fee22f0cc4e84bd9a33ec15e89ef91de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.244708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.244987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.245261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.245884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.248385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.249332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1067/2151] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:33.257895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb227a2-5be8-41bf-af06-2935ac217d3d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.258177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb227a2-5be8-41bf-af06-2935ac217d3d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.258452 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb227a2-5be8-41bf-af06-2935ac217d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.280022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3bb227a2-5be8-41bf-af06-2935ac217d3d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:33.281245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3bb227a2-5be8-41bf-af06-2935ac217d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:33.289433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1068/2152] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:33.294247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb227a2-5be8-41bf-af06-2935ac217d3d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.303718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb227a2-5be8-41bf-af06-2935ac217d3d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.308278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.308865 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.309140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.309485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.310739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:33.311108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.312690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.325545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1069/2153] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:32:33.329681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.339653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc656d2-d0ff-4b4f-a56b-56ce730ba2f9 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.344485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.345058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.345336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.345608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.346219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.348286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.363525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1085/2154] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:33 2022] POST /identity/v3/projects => generated 353 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:33.367761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.400919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:33.457983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1070/2155] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:33 2022] POST /identity/v3/projects => generated 355 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:33.460531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91c7c4-d74a-44df-b53c-39a9cd796bde tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.467392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.527055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfd3feeca3d4c0e97816eb6789a54fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fee22f0cc4e84bd9a33ec15e89ef91de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.527571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.527849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.528689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.529331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:33.529636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.531255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.538286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb0276-dea2-4739-bdfd-3b15a612e4d4 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.545701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.546321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.546596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.546903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.551114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.551202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1086/2156] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:33.551279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.586224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.596674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3f995-55e3-4983-8c53-f059796bbe13 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.602147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfd3feeca3d4c0e97816eb6789a54fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fee22f0cc4e84bd9a33ec15e89ef91de', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.602856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.603138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.603411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] PATH_INFO: `/v3/projects/6026c90e9ac44dcf86c7f90a83cab3b4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.604687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6026c90e9ac44dcf86c7f90a83cab3b4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.607202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.668820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1071/2157] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:33 2022] POST /identity/v3/users => generated 278 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:33.673273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.683004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c42c7-364a-4ff7-ad47-fcef566bd71c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.688194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.688766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.689041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.689309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.689954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.691699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.701750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1072/2158] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:33.705619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.715195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa267d-e9f3-4fea-a2b3-607259b20620 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.720986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.721610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.721887 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.722157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.722819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.724428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.738106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1073/2159] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:33.742300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.753275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1df25-422a-419e-989f-fa81cd74d843 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.754793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6026c90e9ac44dcf86c7f90a83cab3b4', 'request_id': 'req-52d481b7-b1f1-45d5-a335-6afe158a9225'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.767126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6026c90e9ac44dcf86c7f90a83cab3b4', 'request_id': 'req-52d481b7-b1f1-45d5-a335-6afe158a9225'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.774532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6026c90e9ac44dcf86c7f90a83cab3b4', 'request_id': 'req-52d481b7-b1f1-45d5-a335-6afe158a9225'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.814504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.815257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.815532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.815802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] PATH_INFO: `/v3/projects/f630f4c2045f4aad9cc5d5157a73d2ab/users/b15769167da94a839130c269142e95c7/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.830171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f630f4c2045f4aad9cc5d5157a73d2ab, user_id=b15769167da94a839130c269142e95c7, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.832936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.835364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:33.877504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] The token cache is being invalidate because project 6026c90e9ac44dcf86c7f90a83cab3b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:33.877917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.878487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-52d481b7-b1f1-45d5-a335-6afe158a9225 tempest-VolumeQuotaClassesTest-137136612 tempest-VolumeQuotaClassesTest-137136612-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:33.880781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1087/2160] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:33 2022] DELETE /identity/v3/projects/6026c90e9ac44dcf86c7f90a83cab3b4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:33.917811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1074/2161] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:33 2022] PUT /identity/v3/projects/f630f4c2045f4aad9cc5d5157a73d2ab/users/b15769167da94a839130c269142e95c7/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:33.934788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb99069-5c67-4daa-9004-b2c3f0de5b88 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.935209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb99069-5c67-4daa-9004-b2c3f0de5b88 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.935613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb99069-5c67-4daa-9004-b2c3f0de5b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.964013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-deb99069-5c67-4daa-9004-b2c3f0de5b88 None None] MFA Rules not processed for user `b15769167da94a839130c269142e95c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:33.965227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-deb99069-5c67-4daa-9004-b2c3f0de5b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.008304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:34.032837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1088/2162] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.037306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.039040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.103363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b8434-bc0a-49fb-9927-981e537ba6ad admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.108178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.108792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.109068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.109335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.110359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.112625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.253330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1075/2163] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/projects => generated 357 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:34.257351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb99069-5c67-4daa-9004-b2c3f0de5b88 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.271208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb99069-5c67-4daa-9004-b2c3f0de5b88 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.333460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.334094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.334371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.334639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.335301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.337237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.442026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1089/2164] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:34.442726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.456683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e473a25-bf3e-4d5d-8b96-557201589156 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.462980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.463705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.464126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.464519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.467131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.468898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.487426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1076/2165] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:34.488551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.500482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af2dec-28a6-45ff-8a09-785470fa075b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.505175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.505833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.506115 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.506387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] PATH_INFO: `/v3/projects/8d34160b14de4de4b5ae6a46e70ec601/users/e60a0457ebce46e59809591a9b9cc8b5/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.526669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d34160b14de4de4b5ae6a46e70ec601, user_id=e60a0457ebce46e59809591a9b9cc8b5, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.529891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.591674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.593377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.593741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1090/2166] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:34 2022] PUT /identity/v3/projects/8d34160b14de4de4b5ae6a46e70ec601/users/e60a0457ebce46e59809591a9b9cc8b5/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:34.597590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.607526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7198f-e388-4c48-a395-5232bd5b6a85 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.669810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.670841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.671089 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.671470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] PATH_INFO: `/v3/users/1e0fe88d979b40b8a570eb7532b21b68` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.672148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e9b96-6166-4a49-942b-cb345efb5e15 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.672617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e0fe88d979b40b8a570eb7532b21b68)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.674673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.676982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.677592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.677873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.678146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.678818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.680498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.693059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1091/2167] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:34.696845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.707643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbd8fb-b058-4ad6-9ece-e73e1b81450d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.712382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d320b1-caa8-4e06-9c8f-521e500dc61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.713026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d320b1-caa8-4e06-9c8f-521e500dc61d admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.713304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d320b1-caa8-4e06-9c8f-521e500dc61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.713579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d320b1-caa8-4e06-9c8f-521e500dc61d admin admin] PATH_INFO: `/v3/projects/8d34160b14de4de4b5ae6a46e70ec601/users/e60a0457ebce46e59809591a9b9cc8b5/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.715582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d320b1-caa8-4e06-9c8f-521e500dc61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d34160b14de4de4b5ae6a46e70ec601, user_id=e60a0457ebce46e59809591a9b9cc8b5, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.719284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d320b1-caa8-4e06-9c8f-521e500dc61d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.731659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1092/2168] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:34 2022] PUT /identity/v3/projects/8d34160b14de4de4b5ae6a46e70ec601/users/e60a0457ebce46e59809591a9b9cc8b5/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:34.742297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.742581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.742932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.772884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] MFA Rules not processed for user `e60a0457ebce46e59809591a9b9cc8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.774096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.792306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:34.842428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e0fe88d979b40b8a570eb7532b21b68'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.843296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e0fe88d979b40b8a570eb7532b21b68'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.844129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e0fe88d979b40b8a570eb7532b21b68'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.851253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e0fe88d979b40b8a570eb7532b21b68'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.855486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1093/2169] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:34.864715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.866116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.869898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:34.872976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d67b9eaa-af83-40e1-8b3a-ea03d49a3ed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e0fe88d979b40b8a570eb7532b21b68'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.926318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b01df-5d73-4839-bed0-da8627f4c23f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.930153 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.930831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.931057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.931326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.935500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.940459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1077/2170] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:34 2022] DELETE /identity/v3/users/1e0fe88d979b40b8a570eb7532b21b68 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:34.945229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.945547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.945851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.953951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.957154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.965832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.967092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.972136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1094/2171] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:35.035097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1078/2172] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:35.045553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.047214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.110889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999f39-9a34-4467-986b-b2c1175c3ea8 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.170411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.171133 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.171477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.171865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.182811 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.184380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.195296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1095/2173] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:35.289998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.291435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.358328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8407f-ab3e-47d4-8783-c09f7a758cac None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.361969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.362665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.363039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.363408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.365455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.438339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.439826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.451147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1079/2174] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:35.685404 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.686885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.700864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b370c7-285b-457c-a62e-5b8309c6a986 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.707168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.708003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.708375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.708375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] PATH_INFO: `/v3/projects/1f7f15ba0b694082a3bdc09ce3ca5c56` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.709517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7f15ba0b694082a3bdc09ce3ca5c56)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.711383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.727298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:35.834136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7f15ba0b694082a3bdc09ce3ca5c56', 'request_id': 'req-7df42bd1-5153-4ee3-a02a-afc7a229325d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.839497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7f15ba0b694082a3bdc09ce3ca5c56', 'request_id': 'req-7df42bd1-5153-4ee3-a02a-afc7a229325d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.846335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7f15ba0b694082a3bdc09ce3ca5c56', 'request_id': 'req-7df42bd1-5153-4ee3-a02a-afc7a229325d'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.921321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] The token cache is being invalidate because project 1f7f15ba0b694082a3bdc09ce3ca5c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:35.921606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.922220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.923739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1096/2175] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:35 2022] DELETE /identity/v3/projects/1f7f15ba0b694082a3bdc09ce3ca5c56 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:35.927797 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.944765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91980f03-66af-43ae-a8cd-323a5bb3a1fd service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.007362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:36.020024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.022345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.023456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.023456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] PATH_INFO: `/v3/users/fcfd3feeca3d4c0e97816eb6789a54fe` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.025441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcfd3feeca3d4c0e97816eb6789a54fe)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.027756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.282497 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcfd3feeca3d4c0e97816eb6789a54fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:36.283190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcfd3feeca3d4c0e97816eb6789a54fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:36.283831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcfd3feeca3d4c0e97816eb6789a54fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:36.290593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcfd3feeca3d4c0e97816eb6789a54fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:36.303302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-e66f5664-f234-456f-bbd7-248b2122f7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcfd3feeca3d4c0e97816eb6789a54fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:36.396831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1080/2176] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:35 2022] DELETE /identity/v3/users/fcfd3feeca3d4c0e97816eb6789a54fe => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:36.980837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.982265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:36.997206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42bd1-5153-4ee3-a02a-afc7a229325d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.070529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.071200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.071507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.071782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] PATH_INFO: `/v3/projects/fee22f0cc4e84bd9a33ec15e89ef91de` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.072909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee22f0cc4e84bd9a33ec15e89ef91de)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.074838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.180381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee22f0cc4e84bd9a33ec15e89ef91de', 'request_id': 'req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.186430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee22f0cc4e84bd9a33ec15e89ef91de', 'request_id': 'req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.192869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee22f0cc4e84bd9a33ec15e89ef91de', 'request_id': 'req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.275896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] The token cache is being invalidate because project fee22f0cc4e84bd9a33ec15e89ef91de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:37.276285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.276919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a6c70403-b5e7-4a4b-9f15-c33da1d766f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.278413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1097/2177] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:36 2022] DELETE /identity/v3/projects/fee22f0cc4e84bd9a33ec15e89ef91de => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:37.283969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568a3ff-0810-4b77-9d26-5374bb69f8e5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.284435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568a3ff-0810-4b77-9d26-5374bb69f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.284542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568a3ff-0810-4b77-9d26-5374bb69f8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.305363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5568a3ff-0810-4b77-9d26-5374bb69f8e5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.306815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5568a3ff-0810-4b77-9d26-5374bb69f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.370538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1081/2178] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.378952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4e5a-82df-4f4a-a9e3-4a0ddef1d4f7 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.379237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4e5a-82df-4f4a-a9e3-4a0ddef1d4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.379516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a4e5a-82df-4f4a-a9e3-4a0ddef1d4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.407300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4c1a4e5a-82df-4f4a-a9e3-4a0ddef1d4f7 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.408361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4c1a4e5a-82df-4f4a-a9e3-4a0ddef1d4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.427317 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1098/2179] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.431768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568a3ff-0810-4b77-9d26-5374bb69f8e5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.442744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568a3ff-0810-4b77-9d26-5374bb69f8e5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.447555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.448163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.448441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.448708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.449378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:37.449675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.451402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.463575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1082/2180] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:37.468736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a4e5a-82df-4f4a-a9e3-4a0ddef1d4f7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.479445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a4e5a-82df-4f4a-a9e3-4a0ddef1d4f7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.484806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.485426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.485699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.486111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.486642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.488586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.616070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1099/2181] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:37.620328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.631480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414f6a-4360-4098-bc14-b7508f93d0e2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.675254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:37.687597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.688227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.688529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.688801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.689421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.691490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.782445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1083/2182] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:37.786406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.791821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:37.797572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8b746-dab8-4379-a628-d3c04e8152ce admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.802824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.803848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.804340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.804816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.806046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.809188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.824979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1100/2183] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:37.829523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.839616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbde299-ce09-4dae-b986-6fa03eaf3f4e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.845519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b41f70-6ff8-453d-bc39-7544dea7188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.846131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b41f70-6ff8-453d-bc39-7544dea7188e admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.846413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b41f70-6ff8-453d-bc39-7544dea7188e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.846714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b41f70-6ff8-453d-bc39-7544dea7188e admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.847361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b41f70-6ff8-453d-bc39-7544dea7188e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.848968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b41f70-6ff8-453d-bc39-7544dea7188e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.860097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1084/2184] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:37.866641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.876575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55993510-0f23-42ea-b7d2-aceb87b2516e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.881385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.882043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.882322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.882596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] PATH_INFO: `/v3/projects/253764eb5aa540ab916494b4deacd3d8/users/e59366d84fee422e892c2db9cb3224d2/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.898156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] RBAC: Authorizing `identity:create_grant(project_id=253764eb5aa540ab916494b4deacd3d8, user_id=e59366d84fee422e892c2db9cb3224d2, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.900915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.990846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1101/2185] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:37 2022] PUT /identity/v3/projects/253764eb5aa540ab916494b4deacd3d8/users/e59366d84fee422e892c2db9cb3224d2/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:37.995973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbed057-698f-4092-9182-469d50adaa2c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.996251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbed057-698f-4092-9182-469d50adaa2c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.996526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbed057-698f-4092-9182-469d50adaa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.016306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0dbed057-698f-4092-9182-469d50adaa2c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:38.017438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0dbed057-698f-4092-9182-469d50adaa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.087508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1085/2186] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:38.097538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.099425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.170616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3d7d6-631b-463c-b184-68cd3b76830a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.177092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.177709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.177990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.178265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.189248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.190719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.200985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1102/2187] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:39.485599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6540-d521-47ef-99fb-d3192973413e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.485599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6540-d521-47ef-99fb-d3192973413e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.485599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6540-d521-47ef-99fb-d3192973413e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.513333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cbfb6540-d521-47ef-99fb-d3192973413e None None] MFA Rules not processed for user `e59366d84fee422e892c2db9cb3224d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:39.514616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cbfb6540-d521-47ef-99fb-d3192973413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:39.584854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1086/2188] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:39.589375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:39.599545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881de50f-b3f2-40cf-8525-52ef790d40d8 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:39.604044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:39.604622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.604896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.605164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.605781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:39.608355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:39.714330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1103/2189] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:39 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:39.718461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb6540-d521-47ef-99fb-d3192973413e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:39.729539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb6540-d521-47ef-99fb-d3192973413e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:39.784767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:39.790832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:39.791424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.791700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.791969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.798080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:39.798080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:39.896583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1087/2190] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:39 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:39.897740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:39.907908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1caa2-b243-4cdc-a15d-ef9cb909a1bb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:39.914047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:39.914047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.914047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.914047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.914543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:39.916026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:39.927323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1104/2191] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:39.931732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:39.942611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2df38-f008-43d2-a04a-4da464e8c448 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:39.947265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:39.947942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.948223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.948499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] PATH_INFO: `/v3/projects/ab91db724eae4c4d92ad2d4aa2f89189/users/e33ae717d3a749a4917bf822bb183808/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.963465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab91db724eae4c4d92ad2d4aa2f89189, user_id=e33ae717d3a749a4917bf822bb183808, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:39.966190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:40.047797 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1088/2192] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:39 2022] PUT /identity/v3/projects/ab91db724eae4c4d92ad2d4aa2f89189/users/e33ae717d3a749a4917bf822bb183808/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:40.051758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:40.061715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207e93a-7476-4f40-803f-8c0c658c4eca admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:40.118212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a59fa2-fbcd-4a69-ae40-e88d9c4c3244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:40.118852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a59fa2-fbcd-4a69-ae40-e88d9c4c3244 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.119131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a59fa2-fbcd-4a69-ae40-e88d9c4c3244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.119403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a59fa2-fbcd-4a69-ae40-e88d9c4c3244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.120072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a59fa2-fbcd-4a69-ae40-e88d9c4c3244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:40.121670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a59fa2-fbcd-4a69-ae40-e88d9c4c3244 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:40.132171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1105/2193] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:40.136336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:40.147060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fc485-aec6-44eb-bbd0-fef040bfbd04 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:40.151595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:40.152234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.152510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.152779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] PATH_INFO: `/v3/projects/ab91db724eae4c4d92ad2d4aa2f89189/users/e33ae717d3a749a4917bf822bb183808/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.155287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab91db724eae4c4d92ad2d4aa2f89189, user_id=e33ae717d3a749a4917bf822bb183808, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:40.157908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:40.167925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1089/2194] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:40 2022] PUT /identity/v3/projects/ab91db724eae4c4d92ad2d4aa2f89189/users/e33ae717d3a749a4917bf822bb183808/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:41.404430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e369f-7024-4f49-9e34-5c66dc7f63b7 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.405039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e369f-7024-4f49-9e34-5c66dc7f63b7 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.405039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e369f-7024-4f49-9e34-5c66dc7f63b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.437387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-716e369f-7024-4f49-9e34-5c66dc7f63b7 None None] MFA Rules not processed for user `e33ae717d3a749a4917bf822bb183808`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:41.437387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-716e369f-7024-4f49-9e34-5c66dc7f63b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:41.458328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:41.511339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1106/2195] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:41.523918 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.525731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:41.587819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d3bb2-fe90-41d5-8374-e16a77894ce8 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.591422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.592015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.592289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.592553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.647554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.648969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.657894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1090/2196] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:47.613995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e369f-7024-4f49-9e34-5c66dc7f63b7 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:47.624499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e369f-7024-4f49-9e34-5c66dc7f63b7 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:47.628367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf326a-a4d7-43a9-83be-5820d5883329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:47.628963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf326a-a4d7-43a9-83be-5820d5883329 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:47.629238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf326a-a4d7-43a9-83be-5820d5883329 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:47.629507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf326a-a4d7-43a9-83be-5820d5883329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:47.638693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf326a-a4d7-43a9-83be-5820d5883329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:47.640119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf326a-a4d7-43a9-83be-5820d5883329 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:47.649498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1107/2197] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:51.653726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:53.325725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.327618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.394796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ac9a5-9184-493a-8504-1842c18074d9 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.399334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.399942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.400218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.400487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] PATH_INFO: `/v3/users/530cbd6b8b3545ee85e83193749b2ecd` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.401593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=530cbd6b8b3545ee85e83193749b2ecd)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.403532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.601698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530cbd6b8b3545ee85e83193749b2ecd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.602356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530cbd6b8b3545ee85e83193749b2ecd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.602983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530cbd6b8b3545ee85e83193749b2ecd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.612904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530cbd6b8b3545ee85e83193749b2ecd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.627420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530cbd6b8b3545ee85e83193749b2ecd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.704872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1091/2198] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:53 2022] DELETE /identity/v3/users/530cbd6b8b3545ee85e83193749b2ecd => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:53.709203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.709678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.709771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.729371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:53.736617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.790085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1108/2199] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:53.799818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.801314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.860884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769616ec-96a1-4bcc-aa57-81e180bd7f7e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.864728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54d36e-cac5-4870-9741-c88703dcf320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.865342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54d36e-cac5-4870-9741-c88703dcf320 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.865618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54d36e-cac5-4870-9741-c88703dcf320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.865922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54d36e-cac5-4870-9741-c88703dcf320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.875766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54d36e-cac5-4870-9741-c88703dcf320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.877235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54d36e-cac5-4870-9741-c88703dcf320 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.886708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1092/2200] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:54.328376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.329828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.342673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e3274-e425-457e-9d0c-7f6af32b7b05 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.347343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.348035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.348312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.348579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] PATH_INFO: `/v3/projects/230b48971bab4b299b87a633641f7b75` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.349687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230b48971bab4b299b87a633641f7b75)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.351668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.493121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230b48971bab4b299b87a633641f7b75', 'request_id': 'req-abb1f95c-49e4-40ee-b991-0c8109deab15'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.500709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230b48971bab4b299b87a633641f7b75', 'request_id': 'req-abb1f95c-49e4-40ee-b991-0c8109deab15'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.507408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230b48971bab4b299b87a633641f7b75', 'request_id': 'req-abb1f95c-49e4-40ee-b991-0c8109deab15'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.621545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] The token cache is being invalidate because project 230b48971bab4b299b87a633641f7b75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:54.622385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.623020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-abb1f95c-49e4-40ee-b991-0c8109deab15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.624690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1109/2201] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:54 2022] DELETE /identity/v3/projects/230b48971bab4b299b87a633641f7b75 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:54.630411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fbdde-767a-41a3-a6da-a084d5345492 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.630897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fbdde-767a-41a3-a6da-a084d5345492 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.631023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fbdde-767a-41a3-a6da-a084d5345492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.651260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-586fbdde-767a-41a3-a6da-a084d5345492 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:54.652449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-586fbdde-767a-41a3-a6da-a084d5345492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.670443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:54.711255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1093/2202] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:54.719938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad181d7-c4ae-479e-a604-37a544fd722d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.720495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad181d7-c4ae-479e-a604-37a544fd722d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.720618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad181d7-c4ae-479e-a604-37a544fd722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.740288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8ad181d7-c4ae-479e-a604-37a544fd722d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:54.741479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8ad181d7-c4ae-479e-a604-37a544fd722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.749968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1110/2203] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:54.754274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fbdde-767a-41a3-a6da-a084d5345492 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.763932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fbdde-767a-41a3-a6da-a084d5345492 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.768405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.768992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.769270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.769540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.770178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:54.770474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.772079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.788730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1094/2204] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:54.789091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad181d7-c4ae-479e-a604-37a544fd722d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.799085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad181d7-c4ae-479e-a604-37a544fd722d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.803900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.804487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.804762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.805025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.806247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.808263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.923105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:54.931469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1111/2205] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:54.935608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.945793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a3ce6-465a-4c06-bd50-af3c7a9bf016 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.000231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.000819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.001093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.001359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.001961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.004028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.122637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1095/2206] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:55.126784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.138549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aae9f-b0f9-4eb5-aaec-32219188a952 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.144049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.144049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.144241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.144438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.145072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.147367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.168412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1112/2207] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:55.172418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.190246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc623-4a0e-474f-a015-8c9807528b22 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.193873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd6d3c-28e3-4dc6-a5cd-d085f567b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.194593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6d3c-28e3-4dc6-a5cd-d085f567b006 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.194909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6d3c-28e3-4dc6-a5cd-d085f567b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.195185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6d3c-28e3-4dc6-a5cd-d085f567b006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.196437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd6d3c-28e3-4dc6-a5cd-d085f567b006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.198071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd6d3c-28e3-4dc6-a5cd-d085f567b006 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.214350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1096/2208] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:55.215604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.227242 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7bbde-ca12-418d-84ff-2a7866f292cc admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.232902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.233604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.234503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.234503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] PATH_INFO: `/v3/projects/d3d75e4de32444e2813b11b29553d47a/users/1eb6acf487864a41ba1628e9093b8773/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.251032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d75e4de32444e2813b11b29553d47a, user_id=1eb6acf487864a41ba1628e9093b8773, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.253726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.355662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1113/2209] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:55 2022] PUT /identity/v3/projects/d3d75e4de32444e2813b11b29553d47a/users/1eb6acf487864a41ba1628e9093b8773/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:55.361244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.361527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.361798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.380675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:55.381860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.441792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1097/2210] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:55.452040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.453707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.510243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd11c3-ca1f-41be-89ab-df1a11d9279a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.514486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433209c5-c721-4ab6-b423-f4bf3638a9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.515103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433209c5-c721-4ab6-b423-f4bf3638a9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.515383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433209c5-c721-4ab6-b423-f4bf3638a9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.515656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433209c5-c721-4ab6-b423-f4bf3638a9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.524667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433209c5-c721-4ab6-b423-f4bf3638a9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.526159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433209c5-c721-4ab6-b423-f4bf3638a9a1 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.536150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1114/2211] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:58.015030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.016550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.029854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3f39d-510c-4f3b-a56f-6e922fd93645 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.035215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.035868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.036148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.036424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] PATH_INFO: `/v3/users/3ab1e44ca181489b8a4b19ef7fda9844` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.037654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab1e44ca181489b8a4b19ef7fda9844)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.046955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.065201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:58.316192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab1e44ca181489b8a4b19ef7fda9844'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.317361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab1e44ca181489b8a4b19ef7fda9844'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.318125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab1e44ca181489b8a4b19ef7fda9844'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.325950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab1e44ca181489b8a4b19ef7fda9844'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.340311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab1e44ca181489b8a4b19ef7fda9844'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.406950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1098/2212] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:58 2022] DELETE /identity/v3/users/3ab1e44ca181489b8a4b19ef7fda9844 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:58.411534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.412346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.412794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.459662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:58.460901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.542603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1115/2213] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:58.551836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.553317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.616718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244b65e-6f3a-4098-beeb-34d8cbba9f9d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.621061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2eef9-7f9e-47aa-8cd8-5e40c65f0a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.622432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2eef9-7f9e-47aa-8cd8-5e40c65f0a69 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.622854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2eef9-7f9e-47aa-8cd8-5e40c65f0a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.623186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2eef9-7f9e-47aa-8cd8-5e40c65f0a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.636750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2eef9-7f9e-47aa-8cd8-5e40c65f0a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.638510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2eef9-7f9e-47aa-8cd8-5e40c65f0a69 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.649364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1099/2214] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:59.361758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.363327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.376442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08158b34-99ea-4a67-abcf-555dd02811d9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.381265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.381952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.383476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.383476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] PATH_INFO: `/v3/projects/d712579d3d9a45c18768b52cee48cbd9` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.384031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d712579d3d9a45c18768b52cee48cbd9)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.386794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.529582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd712579d3d9a45c18768b52cee48cbd9', 'request_id': 'req-c75d79a7-fde8-4aae-9da1-80763bce638a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:59.537084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd712579d3d9a45c18768b52cee48cbd9', 'request_id': 'req-c75d79a7-fde8-4aae-9da1-80763bce638a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:59.544468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd712579d3d9a45c18768b52cee48cbd9', 'request_id': 'req-c75d79a7-fde8-4aae-9da1-80763bce638a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:59.548312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:59.650948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] The token cache is being invalidate because project d712579d3d9a45c18768b52cee48cbd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:59.650948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:59.651185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c75d79a7-fde8-4aae-9da1-80763bce638a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:59.653229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1116/2215] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:59 2022] DELETE /identity/v3/projects/d712579d3d9a45c18768b52cee48cbd9 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:59.658391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63094582-cdd8-4023-8412-9ed966e5a9af None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.659008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63094582-cdd8-4023-8412-9ed966e5a9af None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.659245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63094582-cdd8-4023-8412-9ed966e5a9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.726367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-63094582-cdd8-4023-8412-9ed966e5a9af None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:59.729087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-63094582-cdd8-4023-8412-9ed966e5a9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.886967 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1100/2216] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:59.898916 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f545f-5c8c-44b8-ba94-01eb8715e312 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.899217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f545f-5c8c-44b8-ba94-01eb8715e312 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.899510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f545f-5c8c-44b8-ba94-01eb8715e312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.939945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ab2f545f-5c8c-44b8-ba94-01eb8715e312 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:59.942107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ab2f545f-5c8c-44b8-ba94-01eb8715e312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.965580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1117/2217] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:59.968319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63094582-cdd8-4023-8412-9ed966e5a9af None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.983184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63094582-cdd8-4023-8412-9ed966e5a9af None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.999639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.000274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.000695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.001525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.002367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:00.002545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.004465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.020516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1101/2218] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:00.025187 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f545f-5c8c-44b8-ba94-01eb8715e312 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.045361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f545f-5c8c-44b8-ba94-01eb8715e312 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.053096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.053803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.054176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.054520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.055267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.057262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.199448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1118/2219] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:00.200177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.211657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabac963-08e2-4592-91a5-872470ec97be admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.301186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:00.336642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.337647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.338083 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.338511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.339574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.343108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.482726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1102/2220] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/users => generated 286 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:00.487163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.500098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518bf2d-b00a-4efb-8d4b-3ee608f788ae admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.505865 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.511480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.511480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.511480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.511480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.511480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.521938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1119/2221] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:00.526046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.538488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e30bf-44ac-466d-af05-7330b1457498 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.544431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e439f2-1cc0-411f-a2b6-ce33f2e65e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.546125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e439f2-1cc0-411f-a2b6-ce33f2e65e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.546125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e439f2-1cc0-411f-a2b6-ce33f2e65e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.546125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e439f2-1cc0-411f-a2b6-ce33f2e65e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.546125 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e439f2-1cc0-411f-a2b6-ce33f2e65e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.547903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e439f2-1cc0-411f-a2b6-ce33f2e65e82 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.559880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1103/2222] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:00.566354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.580253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cfea0-4f96-4dfc-8357-aa98b5640287 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.617569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.618680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.618680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.618897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] PATH_INFO: `/v3/projects/76a93016c5f843ad991cfbf345dc0e22/users/2b99cde76bae4ac4ae73f4cdaaa1d8fe/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.672494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a93016c5f843ad991cfbf345dc0e22, user_id=2b99cde76bae4ac4ae73f4cdaaa1d8fe, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.678147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.777336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1120/2223] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:00 2022] PUT /identity/v3/projects/76a93016c5f843ad991cfbf345dc0e22/users/2b99cde76bae4ac4ae73f4cdaaa1d8fe/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:00.787745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78ba3-009d-4961-88fa-c2168fa96d1a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.788614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78ba3-009d-4961-88fa-c2168fa96d1a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.789047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78ba3-009d-4961-88fa-c2168fa96d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.835049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a6a78ba3-009d-4961-88fa-c2168fa96d1a None None] MFA Rules not processed for user `2b99cde76bae4ac4ae73f4cdaaa1d8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:00.836465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a6a78ba3-009d-4961-88fa-c2168fa96d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.967727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1104/2224] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:00.971213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.973086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:01.063383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3822b-12e3-4278-9cae-f500f4017a1d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.072244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6fa5-1b81-44a7-adb0-014b98aa09e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.072869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6fa5-1b81-44a7-adb0-014b98aa09e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.073194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6fa5-1b81-44a7-adb0-014b98aa09e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.073495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6fa5-1b81-44a7-adb0-014b98aa09e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.096148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6fa5-1b81-44a7-adb0-014b98aa09e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.097831 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6fa5-1b81-44a7-adb0-014b98aa09e1 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.118929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1121/2225] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:01.139175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78ba3-009d-4961-88fa-c2168fa96d1a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.263242 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78ba3-009d-4961-88fa-c2168fa96d1a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.268857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.269543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.270071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.270445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] PATH_INFO: `/v3/users/2b99cde76bae4ac4ae73f4cdaaa1d8fe` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.274621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b99cde76bae4ac4ae73f4cdaaa1d8fe)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.276750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.529266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:01.536029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b99cde76bae4ac4ae73f4cdaaa1d8fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:01.536821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b99cde76bae4ac4ae73f4cdaaa1d8fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:01.537484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b99cde76bae4ac4ae73f4cdaaa1d8fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:01.566757 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b99cde76bae4ac4ae73f4cdaaa1d8fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:01.597038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b99cde76bae4ac4ae73f4cdaaa1d8fe'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:01.688944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1105/2226] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:01 2022] DELETE /identity/v3/users/2b99cde76bae4ac4ae73f4cdaaa1d8fe => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:01.691220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93751c-5114-46fe-985c-56e039b7d765 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.691677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93751c-5114-46fe-985c-56e039b7d765 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.691946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93751c-5114-46fe-985c-56e039b7d765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.733189 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ce93751c-5114-46fe-985c-56e039b7d765 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:01.737121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ce93751c-5114-46fe-985c-56e039b7d765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:01.835081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1122/2227] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:01.847347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.849362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:01.924338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd1642-c0e4-4e41-8207-8eb2e71d281c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.928944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.929570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.929842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.930109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.940107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.945231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.952831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1106/2228] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:02.521537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.522261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.522766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.552629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] MFA Rules not processed for user `1eb6acf487864a41ba1628e9093b8773`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:02.559610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.568202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.570066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.577149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:02.590620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d30d18-dfeb-4d21-afe4-d9c88462399f service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.599949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.600665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.601015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.601377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] PATH_INFO: `/v3/projects/76a93016c5f843ad991cfbf345dc0e22` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.602582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a93016c5f843ad991cfbf345dc0e22)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.604697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.653595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1123/2229] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:02.658197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.659737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.675355 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e3817-e0d9-499b-bfc9-e2250a9c9ceb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.680167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.680882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.681215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.681524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.682032 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.684024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.725498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a93016c5f843ad991cfbf345dc0e22', 'request_id': 'req-5c4ce366-0d46-4f37-9602-4e156f404658'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.732336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a93016c5f843ad991cfbf345dc0e22', 'request_id': 'req-5c4ce366-0d46-4f37-9602-4e156f404658'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.738391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a93016c5f843ad991cfbf345dc0e22', 'request_id': 'req-5c4ce366-0d46-4f37-9602-4e156f404658'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.777347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1124/2230] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:02.782019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.847150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] The token cache is being invalidate because project 76a93016c5f843ad991cfbf345dc0e22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:02.847443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.848600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5c4ce366-0d46-4f37-9602-4e156f404658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.850329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1107/2231] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:02 2022] DELETE /identity/v3/projects/76a93016c5f843ad991cfbf345dc0e22 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:02.855074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb51bf4-727e-4afc-9269-97a050c484b8 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.857828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb51bf4-727e-4afc-9269-97a050c484b8 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.858704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb51bf4-727e-4afc-9269-97a050c484b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.873857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8cc05-bd0c-4f61-97a7-0d42a95509b3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.879475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.880552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.881033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.881497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.882308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.884798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.905103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2bb51bf4-727e-4afc-9269-97a050c484b8 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:02.906410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2bb51bf4-727e-4afc-9269-97a050c484b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.914637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1108/2232] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:02.924675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332827d-0f14-4805-adf9-edcd285ad508 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.924954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332827d-0f14-4805-adf9-edcd285ad508 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.925237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332827d-0f14-4805-adf9-edcd285ad508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.946302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3332827d-0f14-4805-adf9-edcd285ad508 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:02.947539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3332827d-0f14-4805-adf9-edcd285ad508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.957473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1109/2233] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:02.961473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332827d-0f14-4805-adf9-edcd285ad508 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.969105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1125/2234] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:02.971707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332827d-0f14-4805-adf9-edcd285ad508 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.972745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.974929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.976051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.976646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.976915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.977177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.977816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:02.978108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.979758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.987400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459dd7-df6c-4650-81f8-5221f9b13ff3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.991813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1110/2235] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:02.992779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.993376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.993651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.993920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.994621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.996191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.996320 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.006774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca2888-7427-42f1-b4da-6359fe8bb468 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.011558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.011922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1126/2236] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:03.012719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.013004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.013423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.013537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.014048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.015951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.023488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b20ee-19f7-4910-a18d-7876175dc2b6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.028438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.029078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.029353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.029623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] PATH_INFO: `/v3/projects/da1848dea4c945d19a8b4bde5f9fd8e7/users/41d9bdfbf33240b4a405e737ffd221b7/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.032171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:03.045092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1848dea4c945d19a8b4bde5f9fd8e7, user_id=41d9bdfbf33240b4a405e737ffd221b7, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.047848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.151396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1111/2237] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/projects => generated 391 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:03.162488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1127/2238] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:03 2022] PUT /identity/v3/projects/da1848dea4c945d19a8b4bde5f9fd8e7/users/41d9bdfbf33240b4a405e737ffd221b7/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:03.163238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.165483 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.233881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:03.246701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37d580-ef9f-4541-9139-a9560ffd68eb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.251041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624ea9a-417d-4183-839d-38b958ff170b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.257335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.257943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.258224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.258493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.259155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.259629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.260325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.260849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.261453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.261579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.263205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.265545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.272407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1128/2239] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:03.280613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.289587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af20c9-4e10-4a2d-adfb-da59d4743590 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.294227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.294885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.295165 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.295432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] PATH_INFO: `/v3/projects/da1848dea4c945d19a8b4bde5f9fd8e7/users/41d9bdfbf33240b4a405e737ffd221b7/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.297312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1848dea4c945d19a8b4bde5f9fd8e7, user_id=41d9bdfbf33240b4a405e737ffd221b7, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.300149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.310927 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1129/2240] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:03 2022] PUT /identity/v3/projects/da1848dea4c945d19a8b4bde5f9fd8e7/users/41d9bdfbf33240b4a405e737ffd221b7/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:03.372796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1112/2241] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:03 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:03.376959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.461440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad7d6d-add8-499f-94b7-519f620445e0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.461748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.462556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.463074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.463519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.464371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.466902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.484774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1130/2242] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:03.486892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.500139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c9375-b6db-49ea-998d-08dfdd699f6c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.505814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cabb8-a224-4bf6-bbac-830283c76c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.506673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cabb8-a224-4bf6-bbac-830283c76c30 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.507112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cabb8-a224-4bf6-bbac-830283c76c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.515902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cabb8-a224-4bf6-bbac-830283c76c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.515902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cabb8-a224-4bf6-bbac-830283c76c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.515902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cabb8-a224-4bf6-bbac-830283c76c30 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.522803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1113/2243] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:03.526836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.547220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d4ad-e264-4331-86d4-d33dd341a1f4 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.553852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.554922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.555664 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.556010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] PATH_INFO: `/v3/projects/05e977af72c24876a00e211227c581e3/users/4602b414b36946e881940916d19d3414/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.579131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e977af72c24876a00e211227c581e3, user_id=4602b414b36946e881940916d19d3414, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.580316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.678762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1131/2244] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:03 2022] PUT /identity/v3/projects/05e977af72c24876a00e211227c581e3/users/4602b414b36946e881940916d19d3414/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:03.689611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514596db-0155-40b5-92e4-8e8c19d24a50 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.689894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514596db-0155-40b5-92e4-8e8c19d24a50 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.690588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514596db-0155-40b5-92e4-8e8c19d24a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.722880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-514596db-0155-40b5-92e4-8e8c19d24a50 None None] MFA Rules not processed for user `4602b414b36946e881940916d19d3414`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:03.724175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-514596db-0155-40b5-92e4-8e8c19d24a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.835239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1114/2245] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:03.843549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.846598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.912395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430bb7-1aaa-4c5d-a03b-24c97a05a08a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.916515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.917126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.917423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.917688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.927067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.928489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.938568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1132/2246] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:09.432067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.432983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.433773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.478732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] MFA Rules not processed for user `41d9bdfbf33240b4a405e737ffd221b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:09.480025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.541832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:09.560676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1115/2247] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:09.575334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.577001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.651845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bffb7d-87e3-4119-b8e7-66f8e0a60919 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.656986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b51ca-da70-443f-8200-5e693eb3c599 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f33e1c573c044f086f99eb128c8f726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.657592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b51ca-da70-443f-8200-5e693eb3c599 service glance] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.657894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b51ca-da70-443f-8200-5e693eb3c599 service glance] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.658141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b51ca-da70-443f-8200-5e693eb3c599 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.659983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b56b51ca-da70-443f-8200-5e693eb3c599 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.665900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:09.767828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b51ca-da70-443f-8200-5e693eb3c599 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.769294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b51ca-da70-443f-8200-5e693eb3c599 service glance] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.779663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1133/2248] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:09.879974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.881394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.960348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f38cce-f913-4d01-84cd-5e891d088272 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.968971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.968971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.968971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.968971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.971284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.984607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.986259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.999777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1116/2249] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:10.067777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80f39-c54c-4331-90a0-47f4de592d95 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.068116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80f39-c54c-4331-90a0-47f4de592d95 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.068586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80f39-c54c-4331-90a0-47f4de592d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.088762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0cd80f39-c54c-4331-90a0-47f4de592d95 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:10.090040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0cd80f39-c54c-4331-90a0-47f4de592d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:10.168703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1134/2250] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:10.228781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.230221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:10.330342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2c114-559b-410d-b7d2-bab71a269ee0 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.335534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ec25c-2980-423d-8f62-1f0804197de2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.336143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ec25c-2980-423d-8f62-1f0804197de2 service swift] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.336419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ec25c-2980-423d-8f62-1f0804197de2 service swift] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.336690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ec25c-2980-423d-8f62-1f0804197de2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.345890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ec25c-2980-423d-8f62-1f0804197de2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.347495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ec25c-2980-423d-8f62-1f0804197de2 service swift] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.356269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1117/2251] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:33:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:33:20.164327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=50 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:20.352361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=69 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:22.384011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.384526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.384621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.405428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] MFA Rules not processed for user `b9252893998a490689497697663ca4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:22.406699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.465364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1135/2252] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:23.107936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bfa77-4765-490a-ac88-7eafbd05acbf None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.108420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bfa77-4765-490a-ac88-7eafbd05acbf None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.108513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bfa77-4765-490a-ac88-7eafbd05acbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.127202 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e47bfa77-4765-490a-ac88-7eafbd05acbf None None] MFA Rules not processed for user `1eb6acf487864a41ba1628e9093b8773`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:23.128411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e47bfa77-4765-490a-ac88-7eafbd05acbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:23.135219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1118/2253] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:25.025056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.026608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.092278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12776118-f028-4bf3-9b5a-9cd508659ecf None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.096794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.097410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.097692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.097962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] PATH_INFO: `/v3/users/b15769167da94a839130c269142e95c7` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.099130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15769167da94a839130c269142e95c7)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.101044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.295117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15769167da94a839130c269142e95c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.301763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15769167da94a839130c269142e95c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.406567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15769167da94a839130c269142e95c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.419968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15769167da94a839130c269142e95c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.420635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15769167da94a839130c269142e95c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.432751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1136/2254] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:25 2022] DELETE /identity/v3/users/b15769167da94a839130c269142e95c7 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:25.437070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.437522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.437628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.456114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:25.457246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.514952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1119/2255] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:25.524791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.526360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.556116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:25.586084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202df511-7965-4b3c-97a8-8f870f5feb14 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.590755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.591986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.592497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.592774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.602486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.604767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.614418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1137/2256] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:26.115480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.116984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:26.130059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02e46e-98cf-4fcb-b8e4-d0c963542585 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.149216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.149859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.150232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.150389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] PATH_INFO: `/v3/projects/f630f4c2045f4aad9cc5d5157a73d2ab` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.151538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f630f4c2045f4aad9cc5d5157a73d2ab)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.153434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.293637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f630f4c2045f4aad9cc5d5157a73d2ab', 'request_id': 'req-a22dfee5-0a1a-49b3-ba4f-795e6261b442'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.304650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f630f4c2045f4aad9cc5d5157a73d2ab', 'request_id': 'req-a22dfee5-0a1a-49b3-ba4f-795e6261b442'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.311537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f630f4c2045f4aad9cc5d5157a73d2ab', 'request_id': 'req-a22dfee5-0a1a-49b3-ba4f-795e6261b442'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.423032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] The token cache is being invalidate because project f630f4c2045f4aad9cc5d5157a73d2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:26.423413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.424165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.428008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1120/2257] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:26 2022] DELETE /identity/v3/projects/f630f4c2045f4aad9cc5d5157a73d2ab => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:26.430046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.441616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54394e38-d285-4d45-acf7-d41c46e91f50 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.501745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.502369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.502640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.502929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] PATH_INFO: `/v3/users/e60a0457ebce46e59809591a9b9cc8b5` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.504061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60a0457ebce46e59809591a9b9cc8b5)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.506056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.720699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60a0457ebce46e59809591a9b9cc8b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.721353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60a0457ebce46e59809591a9b9cc8b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.809308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60a0457ebce46e59809591a9b9cc8b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.821989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60a0457ebce46e59809591a9b9cc8b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.822578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-7b6eb560-b179-47de-8881-de182060214e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60a0457ebce46e59809591a9b9cc8b5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.834842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1138/2258] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:26 2022] DELETE /identity/v3/users/e60a0457ebce46e59809591a9b9cc8b5 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:27.308280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.309731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.326465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dfee5-0a1a-49b3-ba4f-795e6261b442 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.389133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.389759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.390040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.390309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] PATH_INFO: `/v3/projects/8d34160b14de4de4b5ae6a46e70ec601` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.391457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d34160b14de4de4b5ae6a46e70ec601)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.393559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.418729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:27.502563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d34160b14de4de4b5ae6a46e70ec601', 'request_id': 'req-2329808f-b5df-40fd-9048-85411f38cfba'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.508693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d34160b14de4de4b5ae6a46e70ec601', 'request_id': 'req-2329808f-b5df-40fd-9048-85411f38cfba'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.515571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d34160b14de4de4b5ae6a46e70ec601', 'request_id': 'req-2329808f-b5df-40fd-9048-85411f38cfba'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.671573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] The token cache is being invalidate because project 8d34160b14de4de4b5ae6a46e70ec601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:27.671859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.672487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2329808f-b5df-40fd-9048-85411f38cfba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.674185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1121/2259] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:27 2022] DELETE /identity/v3/projects/8d34160b14de4de4b5ae6a46e70ec601 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:27.678808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf976e-b90a-47cc-94dc-dcd7fc14ff63 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.679586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf976e-b90a-47cc-94dc-dcd7fc14ff63 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.679864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf976e-b90a-47cc-94dc-dcd7fc14ff63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.699911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-48bf976e-b90a-47cc-94dc-dcd7fc14ff63 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:27.701749 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-48bf976e-b90a-47cc-94dc-dcd7fc14ff63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.743473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:27.760899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1139/2260] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.770774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adc24b-8134-41a9-954e-413337eea9db None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.770984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adc24b-8134-41a9-954e-413337eea9db None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.771260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adc24b-8134-41a9-954e-413337eea9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.791545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-00adc24b-8134-41a9-954e-413337eea9db None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:27.792749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-00adc24b-8134-41a9-954e-413337eea9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.801172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1122/2261] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.807121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf976e-b90a-47cc-94dc-dcd7fc14ff63 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.820701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf976e-b90a-47cc-94dc-dcd7fc14ff63 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.825415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.826010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.826284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.826551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.827231 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:27.827580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.829113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.840434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1140/2262] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:27.844895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adc24b-8134-41a9-954e-413337eea9db None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.855361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adc24b-8134-41a9-954e-413337eea9db None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.860567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.861168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.861428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.861782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.862398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.864571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.983713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1123/2263] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:27.987792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.998525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7011ae-a67f-418b-a3f6-69e10fab52d1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.052496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.053089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.053368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.053639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.054248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.056200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.208616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1141/2264] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:28.216740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.226986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5740-62ce-4891-a159-ccabcfef1de5 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.234763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.235401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.235674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.235965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.236783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.239460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.250206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1124/2265] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.254169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.264305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae5ee6-72ee-474f-9886-223316664c8b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.268885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48727a2-4b56-43d3-ab00-72f7e689cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.269468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48727a2-4b56-43d3-ab00-72f7e689cf90 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.269750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48727a2-4b56-43d3-ab00-72f7e689cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.276144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48727a2-4b56-43d3-ab00-72f7e689cf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.276144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48727a2-4b56-43d3-ab00-72f7e689cf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.276144 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48727a2-4b56-43d3-ab00-72f7e689cf90 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.283022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1142/2266] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.287141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.298038 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200e484-1521-4f4d-837a-5393670bf48e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.302765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.308093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.308093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.308093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] PATH_INFO: `/v3/projects/04272e00189c4ef0b56bc2732618ed15/users/e8df6d4ea0414a018e761dc8c298cee3/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.319314 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04272e00189c4ef0b56bc2732618ed15, user_id=e8df6d4ea0414a018e761dc8c298cee3, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.321980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.410742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1125/2267] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:28 2022] PUT /identity/v3/projects/04272e00189c4ef0b56bc2732618ed15/users/e8df6d4ea0414a018e761dc8c298cee3/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:28.421114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c4a5-9d82-46db-a907-136318cd2e50 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.421400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c4a5-9d82-46db-a907-136318cd2e50 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.421856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c4a5-9d82-46db-a907-136318cd2e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.455027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-80f1c4a5-9d82-46db-a907-136318cd2e50 None None] MFA Rules not processed for user `e8df6d4ea0414a018e761dc8c298cee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:28.456266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-80f1c4a5-9d82-46db-a907-136318cd2e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.520058 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1143/2268] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:28.524560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.594175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac06bbb-3131-40a0-8840-c6b88eca6737 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.598913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.599514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.599790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.600057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.600682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.602593 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.716598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1126/2269] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:28.720590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c4a5-9d82-46db-a907-136318cd2e50 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.732176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c4a5-9d82-46db-a907-136318cd2e50 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.784518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:28.802219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.802857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.803134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.803394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.804030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.806210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.912157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1144/2270] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:28.916369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.927473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3e42f-8ddf-43c3-ada8-bc762642a1c6 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.932096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.932685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.932962 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.933282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.933926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.936115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.947643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1127/2271] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.952210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.962035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84597329-2557-4cbb-96bf-6bca4f64caf7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.972227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.972904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.978154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.978154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] PATH_INFO: `/v3/projects/0e9af39e7e524ca3a39209c7295dfe8e/users/3b7f0898227d4a54980b081fa99cb635/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.989072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9af39e7e524ca3a39209c7295dfe8e, user_id=3b7f0898227d4a54980b081fa99cb635, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.992175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.083347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1145/2272] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:28 2022] PUT /identity/v3/projects/0e9af39e7e524ca3a39209c7295dfe8e/users/3b7f0898227d4a54980b081fa99cb635/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.087293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.097893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b16ef-9e9d-47ea-b7a1-e34ae946141a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.110531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:29.154168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be248f6d-ceaa-4cdd-8202-eec5ba07ee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.154853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be248f6d-ceaa-4cdd-8202-eec5ba07ee5e admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.155129 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be248f6d-ceaa-4cdd-8202-eec5ba07ee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.155396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be248f6d-ceaa-4cdd-8202-eec5ba07ee5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.156022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be248f6d-ceaa-4cdd-8202-eec5ba07ee5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.157599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be248f6d-ceaa-4cdd-8202-eec5ba07ee5e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.167265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1128/2273] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:29.172035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.182438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b055a-b8d3-496b-9786-db99e7fcf720 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.187323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.187972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.188248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.188520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] PATH_INFO: `/v3/projects/0e9af39e7e524ca3a39209c7295dfe8e/users/3b7f0898227d4a54980b081fa99cb635/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.190680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9af39e7e524ca3a39209c7295dfe8e, user_id=3b7f0898227d4a54980b081fa99cb635, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.193421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.203659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1146/2274] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:29 2022] PUT /identity/v3/projects/0e9af39e7e524ca3a39209c7295dfe8e/users/3b7f0898227d4a54980b081fa99cb635/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.214025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2029-546b-4164-9215-0b3b0192c694 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.216456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2029-546b-4164-9215-0b3b0192c694 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.216456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2029-546b-4164-9215-0b3b0192c694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.248329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e38b2029-546b-4164-9215-0b3b0192c694 None None] MFA Rules not processed for user `3b7f0898227d4a54980b081fa99cb635`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:29.250370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e38b2029-546b-4164-9215-0b3b0192c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.336448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1129/2275] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:29.347657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.349406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.408628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4de0-4408-4c14-8654-376041939f61 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.412223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45db0fe-0a36-49a8-ba61-89e166f9e3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.412817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45db0fe-0a36-49a8-ba61-89e166f9e3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.413089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45db0fe-0a36-49a8-ba61-89e166f9e3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.413356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45db0fe-0a36-49a8-ba61-89e166f9e3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.425665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45db0fe-0a36-49a8-ba61-89e166f9e3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.427131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45db0fe-0a36-49a8-ba61-89e166f9e3d5 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.436968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1147/2276] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:29.506621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b2029-546b-4164-9215-0b3b0192c694 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.569671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b2029-546b-4164-9215-0b3b0192c694 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.574564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.575233 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.575513 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.575786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] PATH_INFO: `/v3/users/e8df6d4ea0414a018e761dc8c298cee3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.576990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8df6d4ea0414a018e761dc8c298cee3)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.578963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.774251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:29.785316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8df6d4ea0414a018e761dc8c298cee3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.785987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8df6d4ea0414a018e761dc8c298cee3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.786822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8df6d4ea0414a018e761dc8c298cee3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.795397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8df6d4ea0414a018e761dc8c298cee3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.812410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8df6d4ea0414a018e761dc8c298cee3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.889669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1130/2277] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:29 2022] DELETE /identity/v3/users/e8df6d4ea0414a018e761dc8c298cee3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.894239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.894751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.894844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.915510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:29.916816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.980221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1148/2278] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:29.992235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.993872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.055893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe282-da91-4cc5-b35a-c83457843cff admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.059662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.060295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.060571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.060845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.070947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.072374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.083165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1131/2279] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:30.548460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.550021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.562064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f950133-321f-4590-80ba-5227ae25c747 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.566908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.567897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.568181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.568613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] PATH_INFO: `/v3/projects/04272e00189c4ef0b56bc2732618ed15` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.569759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04272e00189c4ef0b56bc2732618ed15)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.572800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.672737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:30.683983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04272e00189c4ef0b56bc2732618ed15', 'request_id': 'req-3ff1da44-c53d-4508-9916-5a77dceb1520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.689311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04272e00189c4ef0b56bc2732618ed15', 'request_id': 'req-3ff1da44-c53d-4508-9916-5a77dceb1520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.695360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04272e00189c4ef0b56bc2732618ed15', 'request_id': 'req-3ff1da44-c53d-4508-9916-5a77dceb1520'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.771673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] The token cache is being invalidate because project 04272e00189c4ef0b56bc2732618ed15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:30.771974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.772633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.774400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1149/2280] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:30 2022] DELETE /identity/v3/projects/04272e00189c4ef0b56bc2732618ed15 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:30.779123 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.788671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794f114-7566-48a9-9d7c-f1d328797709 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.850737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.851360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.851636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.851935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] PATH_INFO: `/v3/users/3b7f0898227d4a54980b081fa99cb635` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.853151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7f0898227d4a54980b081fa99cb635)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.855127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.039736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7f0898227d4a54980b081fa99cb635'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.041067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7f0898227d4a54980b081fa99cb635'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.041067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7f0898227d4a54980b081fa99cb635'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.048746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7f0898227d4a54980b081fa99cb635'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.053820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:31.061050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-074262d3-ade2-4a49-98d4-0bc9a3676538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7f0898227d4a54980b081fa99cb635'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.126013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1132/2281] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:30 2022] DELETE /identity/v3/users/3b7f0898227d4a54980b081fa99cb635 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:31.534631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.536112 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.556025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1da44-c53d-4508-9916-5a77dceb1520 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.614769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.615394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.615670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.615983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] PATH_INFO: `/v3/projects/0e9af39e7e524ca3a39209c7295dfe8e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.617100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9af39e7e524ca3a39209c7295dfe8e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.619014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.737262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9af39e7e524ca3a39209c7295dfe8e', 'request_id': 'req-2d95b779-6515-422d-b852-f6d31c28800a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.743040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9af39e7e524ca3a39209c7295dfe8e', 'request_id': 'req-2d95b779-6515-422d-b852-f6d31c28800a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.749126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9af39e7e524ca3a39209c7295dfe8e', 'request_id': 'req-2d95b779-6515-422d-b852-f6d31c28800a'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.842803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] The token cache is being invalidate because project 0e9af39e7e524ca3a39209c7295dfe8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:31.843095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.843720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2d95b779-6515-422d-b852-f6d31c28800a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.845722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1150/2282] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:31 2022] DELETE /identity/v3/projects/0e9af39e7e524ca3a39209c7295dfe8e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:31.851286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a41e61-99d6-4d87-9c12-946a97d25486 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.854183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a41e61-99d6-4d87-9c12-946a97d25486 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.854183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a41e61-99d6-4d87-9c12-946a97d25486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.871308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-68a41e61-99d6-4d87-9c12-946a97d25486 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:31.872546 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-68a41e61-99d6-4d87-9c12-946a97d25486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.949072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1133/2283] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:31.957314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7ac0-3cc3-406a-be1e-1e1376185596 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.957600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7ac0-3cc3-406a-be1e-1e1376185596 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.957870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7ac0-3cc3-406a-be1e-1e1376185596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.977273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9e4e7ac0-3cc3-406a-be1e-1e1376185596 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:31.978461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9e4e7ac0-3cc3-406a-be1e-1e1376185596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.986383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1151/2284] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:31.991310 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a41e61-99d6-4d87-9c12-946a97d25486 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.001538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a41e61-99d6-4d87-9c12-946a97d25486 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.006182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.006803 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.007082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.007352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.008909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:32.008909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.009924 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.022046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1134/2285] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:32.026535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e7ac0-3cc3-406a-be1e-1e1376185596 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.036479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e7ac0-3cc3-406a-be1e-1e1376185596 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.041339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.042098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.042451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.042948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.043773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.046757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.144309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1152/2286] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:32.148602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.159066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066c7b0-6d1f-4417-8e0a-16f34fcb3f60 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.214094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.214716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.214999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.215269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.216399 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.218301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.305860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1135/2287] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:32.309866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.320335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f18b-5ff2-4380-8f12-0924dce63c0d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.325466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.326169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.326449 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.326750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.327389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.329001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.338854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1153/2288] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:32.343335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.353508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b118c-7259-407f-85ae-a8de97d3532f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.358403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfa66b-e717-470b-bde3-4780d2bd0210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.359031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfa66b-e717-470b-bde3-4780d2bd0210 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.359312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfa66b-e717-470b-bde3-4780d2bd0210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.359586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfa66b-e717-470b-bde3-4780d2bd0210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.360225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfa66b-e717-470b-bde3-4780d2bd0210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.361831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfa66b-e717-470b-bde3-4780d2bd0210 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.372961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1136/2289] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:32.377067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.395327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa4edb-eeb7-4084-9d85-78ae3b212118 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.404993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.404993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.404993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.404993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] PATH_INFO: `/v3/projects/7c08f503a4e247fbb4cfd6bf4a50dbf4/users/5d012393126c4fc680f4da05d2442d6f/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.411696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:32.419251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c08f503a4e247fbb4cfd6bf4a50dbf4, user_id=5d012393126c4fc680f4da05d2442d6f, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.422081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.535289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1154/2290] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:32 2022] PUT /identity/v3/projects/7c08f503a4e247fbb4cfd6bf4a50dbf4/users/5d012393126c4fc680f4da05d2442d6f/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:32.546185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e7852-c034-452a-a654-52a03af050f5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.546464 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e7852-c034-452a-a654-52a03af050f5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.546773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e7852-c034-452a-a654-52a03af050f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.580017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-255e7852-c034-452a-a654-52a03af050f5 None None] MFA Rules not processed for user `5d012393126c4fc680f4da05d2442d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:32.580017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-255e7852-c034-452a-a654-52a03af050f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:32.610287 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:32.640993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1137/2291] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:32.645741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.706848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89a860-81ac-4372-a677-6f4f50f2d7cd admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.711594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.712201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.712487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.712893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.713533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.715482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.845195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1155/2292] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/projects => generated 381 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:32.849198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e7852-c034-452a-a654-52a03af050f5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.860416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e7852-c034-452a-a654-52a03af050f5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.914931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.915524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.915798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.916064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.916666 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.918553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.023725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1138/2293] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:33.027748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.038733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df63f3-ce79-4a06-8a58-579c62c4a6b2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.043603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.044834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.045109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.049836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.049836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.049836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.058432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1156/2294] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:33.062438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.072470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8c98b-2478-4596-ad2a-c5a4a22b009a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.077606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.078244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.078524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.078822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] PATH_INFO: `/v3/projects/ab46be46f12b4d04a87362ebc48b5449/users/3d92c648917a478bada4a555643c7679/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.094252 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab46be46f12b4d04a87362ebc48b5449, user_id=3d92c648917a478bada4a555643c7679, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.096969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.169103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1139/2295] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:33 2022] PUT /identity/v3/projects/ab46be46f12b4d04a87362ebc48b5449/users/3d92c648917a478bada4a555643c7679/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:33.172995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.182967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957dd8fa-1552-4003-97cb-0be7fa002d93 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.238126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00e30d-b43b-4df0-8810-6aeb79bd344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.238748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00e30d-b43b-4df0-8810-6aeb79bd344f admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.239028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00e30d-b43b-4df0-8810-6aeb79bd344f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.239565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00e30d-b43b-4df0-8810-6aeb79bd344f admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.240195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00e30d-b43b-4df0-8810-6aeb79bd344f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.241794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00e30d-b43b-4df0-8810-6aeb79bd344f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.251600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1157/2296] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:33.255660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.266579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db820ce-7b10-451e-b2dc-c6800164e0c3 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.271205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.271841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.272117 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.272393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] PATH_INFO: `/v3/projects/ab46be46f12b4d04a87362ebc48b5449/users/3d92c648917a478bada4a555643c7679/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.274264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab46be46f12b4d04a87362ebc48b5449, user_id=3d92c648917a478bada4a555643c7679, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.276920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.287396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1140/2297] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:33 2022] PUT /identity/v3/projects/ab46be46f12b4d04a87362ebc48b5449/users/3d92c648917a478bada4a555643c7679/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:33.298346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5b13c-3369-4e76-bdf3-923b44846f24 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.298635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5b13c-3369-4e76-bdf3-923b44846f24 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.298943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5b13c-3369-4e76-bdf3-923b44846f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.325991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f2a5b13c-3369-4e76-bdf3-923b44846f24 None None] MFA Rules not processed for user `3d92c648917a478bada4a555643c7679`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:33.327739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f2a5b13c-3369-4e76-bdf3-923b44846f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.376204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:33.396340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1158/2298] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:33.407607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.409289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.469265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350139c7-dfb2-421b-957b-c03f7fcf7e38 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.474780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fcf23-55e1-4504-b377-a526c4b4a204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.475397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fcf23-55e1-4504-b377-a526c4b4a204 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.475796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fcf23-55e1-4504-b377-a526c4b4a204 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.476080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fcf23-55e1-4504-b377-a526c4b4a204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.486089 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fcf23-55e1-4504-b377-a526c4b4a204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.487429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fcf23-55e1-4504-b377-a526c4b4a204 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.498761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1141/2299] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:33.599406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5b13c-3369-4e76-bdf3-923b44846f24 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.609193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5b13c-3369-4e76-bdf3-923b44846f24 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.612814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.613430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.613709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.613980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.671938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.672161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.681370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1159/2300] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:35.764947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.764947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.764947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.783560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:35.785154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:35.815804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:35.891893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1142/2301] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:35.907849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.912179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.047683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf4d0e-d2d5-496a-b1b1-cdebc4a7bca1 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.051960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.052719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.053064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.053403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.067094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.069415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.081431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1160/2302] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:33:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:33:37.499300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.500791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.582607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c06c22-022e-4361-a78a-ff46d3e27d87 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.587532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.588109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.588386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.588660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] PATH_INFO: `/v3/users/e59366d84fee422e892c2db9cb3224d2` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.589818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59366d84fee422e892c2db9cb3224d2)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.591925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.834154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59366d84fee422e892c2db9cb3224d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.834838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59366d84fee422e892c2db9cb3224d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.835436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59366d84fee422e892c2db9cb3224d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.842540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59366d84fee422e892c2db9cb3224d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.855416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59366d84fee422e892c2db9cb3224d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.939830 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1143/2303] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:37 2022] DELETE /identity/v3/users/e59366d84fee422e892c2db9cb3224d2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:38.243898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.244670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.251852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.258222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3442f-2498-4f51-96b4-60c59b839e90 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.343278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.344389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.344814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.345240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] PATH_INFO: `/v3/users/5d012393126c4fc680f4da05d2442d6f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.346557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d012393126c4fc680f4da05d2442d6f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.348825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.352509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.362719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f62112-55aa-477f-a978-ba1912076a33 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.371786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.372759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.374391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.374951 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.376379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] PATH_INFO: `/v3/projects/253764eb5aa540ab916494b4deacd3d8` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.378798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253764eb5aa540ab916494b4deacd3d8)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.382706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.601565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253764eb5aa540ab916494b4deacd3d8', 'request_id': 'req-77322b93-1831-477b-8f90-b71f59af7ef3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.611641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253764eb5aa540ab916494b4deacd3d8', 'request_id': 'req-77322b93-1831-477b-8f90-b71f59af7ef3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.615503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d012393126c4fc680f4da05d2442d6f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.616327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d012393126c4fc680f4da05d2442d6f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.627529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253764eb5aa540ab916494b4deacd3d8', 'request_id': 'req-77322b93-1831-477b-8f90-b71f59af7ef3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.737853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d012393126c4fc680f4da05d2442d6f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.769329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d012393126c4fc680f4da05d2442d6f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.770070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-daef477c-9bad-44b4-a76c-7bf0617a474b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d012393126c4fc680f4da05d2442d6f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.785386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1161/2304] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:38 2022] DELETE /identity/v3/users/5d012393126c4fc680f4da05d2442d6f => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:38.789893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.790173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.790443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.810925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:38.812132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.823235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] The token cache is being invalidate because project 253764eb5aa540ab916494b4deacd3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:38.827619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.827619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.827619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1144/2305] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:38 2022] DELETE /identity/v3/projects/253764eb5aa540ab916494b4deacd3d8 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:38.830388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.844774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.887161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1162/2306] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:38.900662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.902508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.928487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77322b93-1831-477b-8f90-b71f59af7ef3 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.933467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.942208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.942208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.942208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] PATH_INFO: `/v3/users/e33ae717d3a749a4917bf822bb183808` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.942208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33ae717d3a749a4917bf822bb183808)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.942208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.973334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9974255-a522-483c-902e-e430fd5d8f10 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.981634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.982293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.982573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.982874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.993479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.994980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.007329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1163/2307] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:39.151897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33ae717d3a749a4917bf822bb183808'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.152466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33ae717d3a749a4917bf822bb183808'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.153062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33ae717d3a749a4917bf822bb183808'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.160077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33ae717d3a749a4917bf822bb183808'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.172297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33ae717d3a749a4917bf822bb183808'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.267920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1145/2308] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:38 2022] DELETE /identity/v3/users/e33ae717d3a749a4917bf822bb183808 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:39.961225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.967807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.994621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b4e0b-83a8-432c-82a4-afd75bed0a92 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.000822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.003336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:40.067995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.068931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.069323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.069702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] PATH_INFO: `/v3/projects/7c08f503a4e247fbb4cfd6bf4a50dbf4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.071800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c08f503a4e247fbb4cfd6bf4a50dbf4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.074681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.087665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:40.134426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c5f4e-a8bc-4eba-b8a3-99a3e4d51886 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.139296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.139927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.140200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.140469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] PATH_INFO: `/v3/projects/ab91db724eae4c4d92ad2d4aa2f89189` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.141701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab91db724eae4c4d92ad2d4aa2f89189)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.143653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.201405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c08f503a4e247fbb4cfd6bf4a50dbf4', 'request_id': 'req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.207571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c08f503a4e247fbb4cfd6bf4a50dbf4', 'request_id': 'req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.215170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c08f503a4e247fbb4cfd6bf4a50dbf4', 'request_id': 'req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.263697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab91db724eae4c4d92ad2d4aa2f89189', 'request_id': 'req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.269960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab91db724eae4c4d92ad2d4aa2f89189', 'request_id': 'req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.277467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab91db724eae4c4d92ad2d4aa2f89189', 'request_id': 'req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.360074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] The token cache is being invalidate because project 7c08f503a4e247fbb4cfd6bf4a50dbf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:40.360368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.361003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.362585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1164/2309] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:39 2022] DELETE /identity/v3/projects/7c08f503a4e247fbb4cfd6bf4a50dbf4 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:40.366617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.368016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:40.442706 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d3f6-e7b4-409c-9604-47cbe46aaf88 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.446971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] The token cache is being invalidate because project ab91db724eae4c4d92ad2d4aa2f89189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:40.455037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.455037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8216b8e9-ecb6-4b7e-bbf7-7a958fdd6810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.455037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1146/2310] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:39 2022] DELETE /identity/v3/projects/ab91db724eae4c4d92ad2d4aa2f89189 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:40.455037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96106899-326d-4490-a7f8-d15bc2b6936d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.465693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96106899-326d-4490-a7f8-d15bc2b6936d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.465970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96106899-326d-4490-a7f8-d15bc2b6936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.478834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:40.492765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-96106899-326d-4490-a7f8-d15bc2b6936d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:40.495344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-96106899-326d-4490-a7f8-d15bc2b6936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:40.508094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.508728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.509008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.509410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] PATH_INFO: `/v3/users/3d92c648917a478bada4a555643c7679` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.510629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d92c648917a478bada4a555643c7679)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.512605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.576124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1147/2311] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:40.584188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322537bb-cff7-4324-a00e-4664edf5ed38 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.584474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322537bb-cff7-4324-a00e-4664edf5ed38 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.584737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322537bb-cff7-4324-a00e-4664edf5ed38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.606920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-322537bb-cff7-4324-a00e-4664edf5ed38 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:40.608671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-322537bb-cff7-4324-a00e-4664edf5ed38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:40.618847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1148/2312] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:40.623771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322537bb-cff7-4324-a00e-4664edf5ed38 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.633533 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322537bb-cff7-4324-a00e-4664edf5ed38 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.641258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.645023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:40.645023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.645023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.665747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1149/2313] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:40.670851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.682999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2e609-35ce-4fda-9d72-b709cb34594f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.688877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.689568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.690057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.690453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.691283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.693294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.745179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d92c648917a478bada4a555643c7679'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.745926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d92c648917a478bada4a555643c7679'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.793367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1150/2314] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:40.800796 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.817961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d92c648917a478bada4a555643c7679'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.834313 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d92c648917a478bada4a555643c7679'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.835171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d92c648917a478bada4a555643c7679'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:40.854728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1165/2315] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:40 2022] DELETE /identity/v3/users/3d92c648917a478bada4a555643c7679 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:40.897113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a4565-9b99-4a4a-af1d-7d765dd5aa0e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.024520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.025169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.025543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.026309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.026591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.030874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.138387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1151/2316] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/users => generated 292 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:41.142925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:41.144448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:41.163523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda7eba-3fe2-4447-a5b8-8c80feb395f3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.168659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.168866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.169287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.169736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.170918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.172549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.187946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1166/2317] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:41.193632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:41.214493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7057356-fc63-4d2d-b4b7-ec863596bd49 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.222895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf45971-2554-4eb7-8f97-306faf6207c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.223547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf45971-2554-4eb7-8f97-306faf6207c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.223857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf45971-2554-4eb7-8f97-306faf6207c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.224136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf45971-2554-4eb7-8f97-306faf6207c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.224845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf45971-2554-4eb7-8f97-306faf6207c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.226596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf45971-2554-4eb7-8f97-306faf6207c3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.255859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1152/2318] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:41.260284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:41.275205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e993369-a771-455d-93ed-528eddb8660e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.281356 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.283806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.283806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.283806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] PATH_INFO: `/v3/projects/e8e99b28bc4744329374f0b330fa44e2/users/5d11c7350fdd4d9a9e36d507c16ee093/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.310046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e99b28bc4744329374f0b330fa44e2, user_id=5d11c7350fdd4d9a9e36d507c16ee093, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.313550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.466122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1167/2319] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:41 2022] PUT /identity/v3/projects/e8e99b28bc4744329374f0b330fa44e2/users/5d11c7350fdd4d9a9e36d507c16ee093/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:41.478415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.478727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.479007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.516432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] MFA Rules not processed for user `5d11c7350fdd4d9a9e36d507c16ee093`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:41.519602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:41.554704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:41.554704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:41.626682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1153/2320] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:41.631435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:41.635854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:41.637389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:41.659367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2eb44-5cb9-437f-bf23-038fe0ab5265 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.665205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.665860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.666136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.666407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] PATH_INFO: `/v3/projects/ab46be46f12b4d04a87362ebc48b5449` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.670784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab46be46f12b4d04a87362ebc48b5449)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.670784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.700836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f59414-938b-4e48-81c3-e87c35cead38 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.707833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.708479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.708906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.709068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.725209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.726854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.738062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1154/2321] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:41.825245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab46be46f12b4d04a87362ebc48b5449', 'request_id': 'req-f8da8d66-382e-411d-affd-9dbe768bdf7c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:41.831628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab46be46f12b4d04a87362ebc48b5449', 'request_id': 'req-f8da8d66-382e-411d-affd-9dbe768bdf7c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:41.838186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab46be46f12b4d04a87362ebc48b5449', 'request_id': 'req-f8da8d66-382e-411d-affd-9dbe768bdf7c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:41.850985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:41.911825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] The token cache is being invalidate because project ab46be46f12b4d04a87362ebc48b5449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:41.914906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:41.914906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:41.914906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1168/2322] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:41 2022] DELETE /identity/v3/projects/ab46be46f12b4d04a87362ebc48b5449 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:42.380270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:42.380270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:42.387373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d9f3-2eff-4f92-9a6b-43abb28ddd8d service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:42.460009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:42.460855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:42.461152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:42.461442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] PATH_INFO: `/v3/users/a0a02bd98d434676a6e0e30ed07f8f8c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:42.465754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a02bd98d434676a6e0e30ed07f8f8c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:42.467846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:42.480431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:42.489913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:42.496271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da8d66-382e-411d-affd-9dbe768bdf7c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:42.501533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:42.502160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:42.502440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:42.502753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] PATH_INFO: `/v3/users/4602b414b36946e881940916d19d3414` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:42.503896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4602b414b36946e881940916d19d3414)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:42.505830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:42.743559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a02bd98d434676a6e0e30ed07f8f8c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.744229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a02bd98d434676a6e0e30ed07f8f8c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.744838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a02bd98d434676a6e0e30ed07f8f8c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.755504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a02bd98d434676a6e0e30ed07f8f8c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.772557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ed61869-f8d4-431f-807c-f09a1e3e4085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a02bd98d434676a6e0e30ed07f8f8c'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.777586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4602b414b36946e881940916d19d3414'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.778419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4602b414b36946e881940916d19d3414'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.886342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1155/2323] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:42 2022] DELETE /identity/v3/users/a0a02bd98d434676a6e0e30ed07f8f8c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:42.890897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:42.891282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:42.891629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:42.892583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:42.926677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:42.926677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:42.946957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4602b414b36946e881940916d19d3414'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.962752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4602b414b36946e881940916d19d3414'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.963580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2731cbfd-ebf2-48b2-b332-4f33de1dffdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4602b414b36946e881940916d19d3414'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:42.988260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1169/2324] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:42 2022] DELETE /identity/v3/users/4602b414b36946e881940916d19d3414 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:42.993105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:42.993966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:42.994336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:42.995961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1156/2325] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:43.005912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.008025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.026211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:43.027641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.080378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32322c4a-d0de-45e7-9990-11eca0aa6b86 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.104199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:43.121219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1170/2326] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:43.131655 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.134377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.170629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.171550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.171839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.172113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.194169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.196831 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.227008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1157/2327] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:43.243328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7cb3e-589d-45a0-bf9b-bdc4171135c4 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.252518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:43.268253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.269075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.269516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.269943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.299567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.301109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.314834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:43.323412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1171/2328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:43.797040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.798794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.820620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b71d2a-4a7b-4af1-8a7d-fbf75712c69c service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.827222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.828122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.828577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.829009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] PATH_INFO: `/v3/projects/f131aa47c61e4c338885741c49918e4c` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.830573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f131aa47c61e4c338885741c49918e4c)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.842351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.978358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.980015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.003588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e4981-8943-4772-b666-01fd40a5b2c0 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.013767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f131aa47c61e4c338885741c49918e4c', 'request_id': 'req-a74f660b-ad1e-4b16-b7d8-eb39683859ea'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.027662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f131aa47c61e4c338885741c49918e4c', 'request_id': 'req-a74f660b-ad1e-4b16-b7d8-eb39683859ea'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.039214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f131aa47c61e4c338885741c49918e4c', 'request_id': 'req-a74f660b-ad1e-4b16-b7d8-eb39683859ea'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.072108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.073044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.073319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.073586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] PATH_INFO: `/v3/projects/05e977af72c24876a00e211227c581e3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.074723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e977af72c24876a00e211227c581e3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.076702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.142885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] The token cache is being invalidate because project f131aa47c61e4c338885741c49918e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:44.143507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.144340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-a74f660b-ad1e-4b16-b7d8-eb39683859ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.146988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1158/2329] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:43 2022] DELETE /identity/v3/projects/f131aa47c61e4c338885741c49918e4c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:44.151853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba105d1-57ab-452e-a2cb-9b5ce451fe08 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.152369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba105d1-57ab-452e-a2cb-9b5ce451fe08 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.152639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba105d1-57ab-452e-a2cb-9b5ce451fe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.176306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-1ba105d1-57ab-452e-a2cb-9b5ce451fe08 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:44.177615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1ba105d1-57ab-452e-a2cb-9b5ce451fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.180236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e977af72c24876a00e211227c581e3', 'request_id': 'req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.187164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e977af72c24876a00e211227c581e3', 'request_id': 'req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.187843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:44.193632 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e977af72c24876a00e211227c581e3', 'request_id': 'req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.255029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1159/2330] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:44.265747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] The token cache is being invalidate because project 05e977af72c24876a00e211227c581e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:44.266195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.267071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-1fd5ca35-1fa4-4cf7-a157-5412c01a060c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.274270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1172/2331] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:43 2022] DELETE /identity/v3/projects/05e977af72c24876a00e211227c581e3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:44.275973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce656-7cc7-4b4c-a4d4-a7c10471347c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.276260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce656-7cc7-4b4c-a4d4-a7c10471347c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.276533 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ce656-7cc7-4b4c-a4d4-a7c10471347c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.303725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a20ce656-7cc7-4b4c-a4d4-a7c10471347c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:44.305091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a20ce656-7cc7-4b4c-a4d4-a7c10471347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.317819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1160/2332] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:44.329421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ad755-db5f-4344-9011-16fbd700ca43 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.329703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ad755-db5f-4344-9011-16fbd700ca43 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.329976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ad755-db5f-4344-9011-16fbd700ca43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.332381 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa9d3-4607-4498-9791-42b1debe9b84 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.332897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa9d3-4607-4498-9791-42b1debe9b84 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.333326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa9d3-4607-4498-9791-42b1debe9b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.357659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-671fa9d3-4607-4498-9791-42b1debe9b84 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:44.358947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-671fa9d3-4607-4498-9791-42b1debe9b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.363189 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7b1ad755-db5f-4344-9011-16fbd700ca43 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:44.364874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7b1ad755-db5f-4344-9011-16fbd700ca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.371384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1161/2333] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:44.372534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa9d3-4607-4498-9791-42b1debe9b84 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.377338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1173/2334] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:44.381675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ad755-db5f-4344-9011-16fbd700ca43 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.383586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa9d3-4607-4498-9791-42b1debe9b84 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.389425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.392465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.392465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.392465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.392465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:44.392465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.394696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.396506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ad755-db5f-4344-9011-16fbd700ca43 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.409734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1162/2335] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:44.413577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.413959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.416285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.416725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.417289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.418090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:44.421963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.424046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.440635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fc9b9-3c79-4407-8dc5-2ef0103fc1cc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.447384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.460283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.460283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.460283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.462235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.462747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1174/2336] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:44.463763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.464499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.475211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88c529-5ca8-46b5-8d67-9893ec0b81b9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.480589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.481524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.481883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.483084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.483779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.485771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.554741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1163/2337] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:44.576795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.639447 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1175/2338] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:44.651713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.657844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24955a7-42d0-4451-9e22-ed900bd949aa admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.736800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.737669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.738225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.738503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.739425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.741346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.760287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:44.774539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3567fe-328b-42b8-bddc-b3006f7480f1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.780039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.780642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.780918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.781213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.782036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.784080 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.842063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1164/2339] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:44.847270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.854996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:44.861890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2077e-3d11-454f-8abd-f52b6058adef admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.867648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.868869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.869098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.875410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.875551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1176/2340] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/users => generated 300 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:44.875656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.875656 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.876847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.883526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1165/2341] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:44.887165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.897418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56da8-c1a3-488e-9781-9d4d96adaae6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.902102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d154d06-e702-4280-b4e7-11d5bfe9eebc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.903110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.904715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.904991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.905418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.909622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.910490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.920903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.920903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.920903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.920903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.921832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.921832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.929878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1177/2342] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:44.934726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1166/2343] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:44.934998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.938441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.951901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f4b4c-1227-4217-8a79-c85738ebbc77 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.954164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c10ce-d69a-4eba-ad0b-90c66282fc09 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.957350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.958030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.958394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.958763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.959502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.960316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de1b2a-7ea7-4c4e-a89e-af9f8b7f3d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.961306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1b2a-7ea7-4c4e-a89e-af9f8b7f3d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.961435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.961839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1b2a-7ea7-4c4e-a89e-af9f8b7f3d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.962270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1b2a-7ea7-4c4e-a89e-af9f8b7f3d0e admin admin] PATH_INFO: `/v3/projects/854059a50fa44d4a9b871259d3871993/users/91ee95a4186d41968c21e201c813787e/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.978864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1178/2344] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:44.984362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.984499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de1b2a-7ea7-4c4e-a89e-af9f8b7f3d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=854059a50fa44d4a9b871259d3871993, user_id=91ee95a4186d41968c21e201c813787e, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.987228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de1b2a-7ea7-4c4e-a89e-af9f8b7f3d0e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.993504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c20171-dfad-4597-9664-bc76403493c5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.998542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b575f-bf9d-4cb7-b0eb-309ffa3db7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.999230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b575f-bf9d-4cb7-b0eb-309ffa3db7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.999507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b575f-bf9d-4cb7-b0eb-309ffa3db7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.999802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b575f-bf9d-4cb7-b0eb-309ffa3db7c9 admin admin] PATH_INFO: `/v3/projects/ad108f789f4f41c892abee270a5c919b/users/7bfc6a0b4dd84f42b34b2b39128494ed/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.016132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b575f-bf9d-4cb7-b0eb-309ffa3db7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad108f789f4f41c892abee270a5c919b, user_id=7bfc6a0b4dd84f42b34b2b39128494ed, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.019783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b575f-bf9d-4cb7-b0eb-309ffa3db7c9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.073006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1167/2345] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:44 2022] PUT /identity/v3/projects/854059a50fa44d4a9b871259d3871993/users/91ee95a4186d41968c21e201c813787e/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:45.083996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.084278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.084582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.110829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1179/2346] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:44 2022] PUT /identity/v3/projects/ad108f789f4f41c892abee270a5c919b/users/7bfc6a0b4dd84f42b34b2b39128494ed/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:45.116199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] MFA Rules not processed for user `91ee95a4186d41968c21e201c813787e`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:45.117416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.125215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.125496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.125763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.158457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] MFA Rules not processed for user `7bfc6a0b4dd84f42b34b2b39128494ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:45.159843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.195807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1168/2347] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:45.219034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.220406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.252576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1180/2348] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:45.282954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0065f45-d76b-456f-aaa1-9d2a4f53a316 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.287143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.287749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.288025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.288326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.297453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.298963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.311616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1169/2349] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:45.399440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.401010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.434217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60185-bcca-4059-b17f-07e0e0f730cd None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.439022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05b9b1-738e-40ad-89e9-d146e37b4e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.451531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05b9b1-738e-40ad-89e9-d146e37b4e6c service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.451531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05b9b1-738e-40ad-89e9-d146e37b4e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.451531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05b9b1-738e-40ad-89e9-d146e37b4e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.452155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05b9b1-738e-40ad-89e9-d146e37b4e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.453719 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05b9b1-738e-40ad-89e9-d146e37b4e6c service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.473668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1181/2350] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:54.839119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:54.886941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:54.900357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c72e5-c272-4478-90e9-cbe02b3e9982 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:54.904861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:54.905467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.905742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.906017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] PATH_INFO: `/v3/users/91ee95a4186d41968c21e201c813787e` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.907136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ee95a4186d41968c21e201c813787e)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:54.909037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.105175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ee95a4186d41968c21e201c813787e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.105834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ee95a4186d41968c21e201c813787e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.106427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ee95a4186d41968c21e201c813787e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.113993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ee95a4186d41968c21e201c813787e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.127731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ee95a4186d41968c21e201c813787e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.199841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1170/2351] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:54 2022] DELETE /identity/v3/users/91ee95a4186d41968c21e201c813787e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:55.204413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.204929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.205040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.224518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:55.225721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.294516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1182/2352] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:55.304570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.306583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.365970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ca297-848e-4bc2-b0f6-46c3a51408ca admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.369652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.370133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.370405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.370699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.380321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.381734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.391624 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1171/2353] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:55.819024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.820462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.832595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaca26b-6b53-4f1d-a875-ac0c6419849f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.837366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.837991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.838307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.838559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] PATH_INFO: `/v3/projects/854059a50fa44d4a9b871259d3871993` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.839721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=854059a50fa44d4a9b871259d3871993)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.841708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.844856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:55.932392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854059a50fa44d4a9b871259d3871993', 'request_id': 'req-0fea99ee-b05e-4b02-aa99-de68fda28e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.937970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854059a50fa44d4a9b871259d3871993', 'request_id': 'req-0fea99ee-b05e-4b02-aa99-de68fda28e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.944147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854059a50fa44d4a9b871259d3871993', 'request_id': 'req-0fea99ee-b05e-4b02-aa99-de68fda28e3c'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.007762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] The token cache is being invalidate because project 854059a50fa44d4a9b871259d3871993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:56.008048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.008760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0fea99ee-b05e-4b02-aa99-de68fda28e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.010278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1183/2354] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:55 2022] DELETE /identity/v3/projects/854059a50fa44d4a9b871259d3871993 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:59.665879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:59.667293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:59.680849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974309b0-a7df-40a5-8614-c4303e0c61c3 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:59.740663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:59.741286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:59.741565 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:59.741837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] PATH_INFO: `/v3/users/5d11c7350fdd4d9a9e36d507c16ee093` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:59.742997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d11c7350fdd4d9a9e36d507c16ee093)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:59.744897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:59.758283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:59.915007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d11c7350fdd4d9a9e36d507c16ee093'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.917373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d11c7350fdd4d9a9e36d507c16ee093'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.917373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d11c7350fdd4d9a9e36d507c16ee093'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.924649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d11c7350fdd4d9a9e36d507c16ee093'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.937414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d11c7350fdd4d9a9e36d507c16ee093'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.024867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1172/2355] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:59 2022] DELETE /identity/v3/users/5d11c7350fdd4d9a9e36d507c16ee093 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:00.030442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.034761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.034761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.050954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:00.052263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.112270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1184/2356] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:00.122977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.124726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.183307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b048f7a-a8d2-4a83-9f67-8b92f52d7bd4 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.187261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da9004-93c5-406a-a82a-e2c06338a9fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.187950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9004-93c5-406a-a82a-e2c06338a9fc service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.188283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9004-93c5-406a-a82a-e2c06338a9fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.188549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9004-93c5-406a-a82a-e2c06338a9fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.197902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da9004-93c5-406a-a82a-e2c06338a9fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.199368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da9004-93c5-406a-a82a-e2c06338a9fc service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.209462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1173/2357] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:00.740061 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.743395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.758609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb53071-35d0-4984-8e8f-eb5c73393c25 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.763781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.764488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.764762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.765030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] PATH_INFO: `/v3/projects/e8e99b28bc4744329374f0b330fa44e2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.766160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e99b28bc4744329374f0b330fa44e2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.769220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.864570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e99b28bc4744329374f0b330fa44e2', 'request_id': 'req-c59d00fa-0f41-44de-b145-f506c108bff9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.870122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e99b28bc4744329374f0b330fa44e2', 'request_id': 'req-c59d00fa-0f41-44de-b145-f506c108bff9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.877996 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e99b28bc4744329374f0b330fa44e2', 'request_id': 'req-c59d00fa-0f41-44de-b145-f506c108bff9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.014641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] The token cache is being invalidate because project e8e99b28bc4744329374f0b330fa44e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:01.014641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.017735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c59d00fa-0f41-44de-b145-f506c108bff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.019846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1185/2358] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:00 2022] DELETE /identity/v3/projects/e8e99b28bc4744329374f0b330fa44e2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:01.025012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab1eea-de96-4f54-8ad6-a2c4d940a162 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.025583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab1eea-de96-4f54-8ad6-a2c4d940a162 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.025583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab1eea-de96-4f54-8ad6-a2c4d940a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.045967 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-8dab1eea-de96-4f54-8ad6-a2c4d940a162 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:01.047188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8dab1eea-de96-4f54-8ad6-a2c4d940a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.108787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1174/2359] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:01.116949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843c02-f892-4772-86f0-fe0f8f5dda94 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.117526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843c02-f892-4772-86f0-fe0f8f5dda94 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.117526 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843c02-f892-4772-86f0-fe0f8f5dda94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.138389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0a843c02-f892-4772-86f0-fe0f8f5dda94 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:01.139651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0a843c02-f892-4772-86f0-fe0f8f5dda94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.147670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1186/2360] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:01.157857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab1eea-de96-4f54-8ad6-a2c4d940a162 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.167820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab1eea-de96-4f54-8ad6-a2c4d940a162 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.172945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.173623 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.173954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.174290 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.175043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:01.175405 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.177182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.189103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1175/2361] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:01.193451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a843c02-f892-4772-86f0-fe0f8f5dda94 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.202991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a843c02-f892-4772-86f0-fe0f8f5dda94 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.207709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.208307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.208586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.208878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.209509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.211650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.217636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:01.307145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1187/2362] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:01.311449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.321370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d1cab-fafd-4f47-85d7-017cd5ad9a9f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.339014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:01.378456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.379172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.379453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.380730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.381379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.383331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.501118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1176/2363] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:01.506027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.516743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51fbe-36ef-45be-9190-34afcd513956 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.521443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.522036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.522305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.522571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.523331 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.524935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.535853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1188/2364] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:01.540133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.550826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60901baa-15eb-4279-af99-e1a493f3a65d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.555738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f3e49-f96c-4893-abc8-f9f440de49f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.556410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f3e49-f96c-4893-abc8-f9f440de49f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.556762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f3e49-f96c-4893-abc8-f9f440de49f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.557136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f3e49-f96c-4893-abc8-f9f440de49f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.557898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f3e49-f96c-4893-abc8-f9f440de49f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.559576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f3e49-f96c-4893-abc8-f9f440de49f2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.570921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1177/2365] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:01.575157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.585844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481d715-5686-4223-9618-01d62bcd0077 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.590478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.591794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.592075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.592348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] PATH_INFO: `/v3/projects/16dcc862ab844cd4a8a241eaf2bd8a91/users/c99b477597e447e98c9e7e9bfc9c19c3/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.611052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16dcc862ab844cd4a8a241eaf2bd8a91, user_id=c99b477597e447e98c9e7e9bfc9c19c3, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.611052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.711995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1189/2366] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:01 2022] PUT /identity/v3/projects/16dcc862ab844cd4a8a241eaf2bd8a91/users/c99b477597e447e98c9e7e9bfc9c19c3/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:01.724828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ba2f-ee90-41e1-9b92-4d8fdccdeff2 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.725108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ba2f-ee90-41e1-9b92-4d8fdccdeff2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.725377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ba2f-ee90-41e1-9b92-4d8fdccdeff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.753031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d1e4ba2f-ee90-41e1-9b92-4d8fdccdeff2 None None] MFA Rules not processed for user `c99b477597e447e98c9e7e9bfc9c19c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:01.754245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d1e4ba2f-ee90-41e1-9b92-4d8fdccdeff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.817128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1178/2367] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:01.821701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.886868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2408-28f4-4b33-a16d-8feaef1ac4d1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.891988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.892573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.892847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.893111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.893716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.895686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.975362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1190/2368] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:01.979514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4ba2f-ee90-41e1-9b92-4d8fdccdeff2 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.990453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4ba2f-ee90-41e1-9b92-4d8fdccdeff2 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.046387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.047160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.047286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.047557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.048189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.050221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.077411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.079178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.091605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238f8c2-3e0a-4d5c-8c87-96fdbc243099 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.096134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.096751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.097029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.097304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] PATH_INFO: `/v3/users/7bfc6a0b4dd84f42b34b2b39128494ed` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.098426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bfc6a0b4dd84f42b34b2b39128494ed)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.100364 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.181030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1179/2369] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:02.184524 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.194740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82093f6d-61e8-4c48-997e-77d7faf9d61d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.199518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.200198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.201263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.201632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.202334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.204023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.214509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1180/2370] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:02.218190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.229721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1eb4f-3db3-4a2d-bb4b-dac552fc8177 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.234333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.234969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.235895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.236175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.236812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.238406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.240437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:02.250339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1181/2371] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:02.254356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.264607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50142849-d092-41ab-b245-da6be6c88aa9 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.269368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78580e-659b-4cf3-86a5-3276311cdfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.270003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78580e-659b-4cf3-86a5-3276311cdfed admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.270279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78580e-659b-4cf3-86a5-3276311cdfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.270547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78580e-659b-4cf3-86a5-3276311cdfed admin admin] PATH_INFO: `/v3/projects/433e4be298f84dbbac46f8cf39bd6966/users/0184bcac64a7422bb4b70b1bd3a56ca5/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.285704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78580e-659b-4cf3-86a5-3276311cdfed admin admin] RBAC: Authorizing `identity:create_grant(project_id=433e4be298f84dbbac46f8cf39bd6966, user_id=0184bcac64a7422bb4b70b1bd3a56ca5, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.288441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78580e-659b-4cf3-86a5-3276311cdfed admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.345690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bfc6a0b4dd84f42b34b2b39128494ed'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:02.346272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bfc6a0b4dd84f42b34b2b39128494ed'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:02.370236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1182/2372] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:02 2022] PUT /identity/v3/projects/433e4be298f84dbbac46f8cf39bd6966/users/0184bcac64a7422bb4b70b1bd3a56ca5/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:02.380933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.381223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.381484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.408896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] MFA Rules not processed for user `0184bcac64a7422bb4b70b1bd3a56ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:02.410258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.446294 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bfc6a0b4dd84f42b34b2b39128494ed'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:02.460617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bfc6a0b4dd84f42b34b2b39128494ed'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:02.461319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6a5b95bd-ab72-47fd-8777-d023e620baf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bfc6a0b4dd84f42b34b2b39128494ed'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:02.477639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1191/2373] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:02 2022] DELETE /identity/v3/users/7bfc6a0b4dd84f42b34b2b39128494ed => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:02.480007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.480297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.480571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.482493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1183/2374] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:02.487592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.489403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.501852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:02.503219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.507525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:02.560522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f4d1-8e4e-473a-969c-3b33e7115cf4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.569679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1192/2375] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:02.579556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.581169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.618063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.618583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.618890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.619165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.619782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.621786 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.637207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:02.639786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba184556-a9e1-4b66-9bc9-55b374899c63 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.643439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.643965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.644239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.644506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.653226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.655232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.664854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1193/2376] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:02.710222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1184/2377] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:02 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:02.712516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.721562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73db61b-7e50-4587-a9ec-a0d6ec9bc270 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.779383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.780031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.787025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.787025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.787025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.787025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.886316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1194/2378] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:02 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:02.890208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.900701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075ae4-d6f1-4d0a-8201-404f84b40a35 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.906077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.906732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.907014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.907288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.907928 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.909540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.921437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1185/2379] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:02.925528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.935812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06c85-9a73-43ba-b3d6-84f4017f4c1b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.940888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.941557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.941886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.942172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] PATH_INFO: `/v3/projects/dc3706438f764234a225b305db6fe569/users/db58d7bc86174fba950a909002e70bd4/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.957882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3706438f764234a225b305db6fe569, user_id=db58d7bc86174fba950a909002e70bd4, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.960484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:03.055002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1195/2380] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:02 2022] PUT /identity/v3/projects/dc3706438f764234a225b305db6fe569/users/db58d7bc86174fba950a909002e70bd4/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:03.058955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:03.068782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d955-6ea4-4b22-9654-5cd4415ea3d7 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:03.125828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:03.126416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:03.126715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:03.127026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:03.127653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:03.129225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:03.139426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1186/2381] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:03.143595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:03.145062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:03.146753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:03.156650 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ebca6-0ee6-4065-aac9-193aa740d8e9 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:03.160241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fb8a2-dd7b-464a-88af-84906cbbf409 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:03.160986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3347a0-f3b9-41d0-91e4-ff867bd5b5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:03.161933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3347a0-f3b9-41d0-91e4-ff867bd5b5b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:03.162216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3347a0-f3b9-41d0-91e4-ff867bd5b5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:03.162492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3347a0-f3b9-41d0-91e4-ff867bd5b5b3 admin admin] PATH_INFO: `/v3/projects/dc3706438f764234a225b305db6fe569/users/db58d7bc86174fba950a909002e70bd4/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:03.165198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3347a0-f3b9-41d0-91e4-ff867bd5b5b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3706438f764234a225b305db6fe569, user_id=db58d7bc86174fba950a909002e70bd4, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:03.165771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:03.166397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:03.166681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:03.166964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] PATH_INFO: `/v3/projects/ad108f789f4f41c892abee270a5c919b` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:03.167930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3347a0-f3b9-41d0-91e4-ff867bd5b5b3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:03.168093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad108f789f4f41c892abee270a5c919b)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:03.170040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:03.179393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1196/2382] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:03 2022] PUT /identity/v3/projects/dc3706438f764234a225b305db6fe569/users/db58d7bc86174fba950a909002e70bd4/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:03.189626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:03.189959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:03.190181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:03.198436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:03.222029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] MFA Rules not processed for user `db58d7bc86174fba950a909002e70bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:03.223390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:03.294504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad108f789f4f41c892abee270a5c919b', 'request_id': 'req-22bdb6bd-62c1-4aea-a19e-ad990dc44331'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:03.304131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad108f789f4f41c892abee270a5c919b', 'request_id': 'req-22bdb6bd-62c1-4aea-a19e-ad990dc44331'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:03.309337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad108f789f4f41c892abee270a5c919b', 'request_id': 'req-22bdb6bd-62c1-4aea-a19e-ad990dc44331'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:03.315057 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1197/2383] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:03.328003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:03.329568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:03.392667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] The token cache is being invalidate because project ad108f789f4f41c892abee270a5c919b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:03.392817 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:03.393491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:03.395383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1187/2384] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:03 2022] DELETE /identity/v3/projects/ad108f789f4f41c892abee270a5c919b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:03.405828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ad605-7229-42c6-a18e-8e364d61a860 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:03.486863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:03.487484 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:03.487758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:03.488021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:03.490324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:03.561972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:03.563627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:03.569233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:03.574207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1198/2385] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:06.598935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:06.600393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:06.613738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb6bd-62c1-4aea-a19e-ad990dc44331 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:06.617617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:06.618209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:06.618490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:06.618813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:06.620495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:06.678040 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:06.679499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:06.684008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:06.688947 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1188/2386] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:08.387502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.399067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e39a3-7122-4151-a292-13a956548aef service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.404776 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc5fdb-0c3a-4011-9875-c1d9c7c3274f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.405495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc5fdb-0c3a-4011-9875-c1d9c7c3274f service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.405876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc5fdb-0c3a-4011-9875-c1d9c7c3274f service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.406245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc5fdb-0c3a-4011-9875-c1d9c7c3274f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.408919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fddc5fdb-0c3a-4011-9875-c1d9c7c3274f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:08.521207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc5fdb-0c3a-4011-9875-c1d9c7c3274f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.522711 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc5fdb-0c3a-4011-9875-c1d9c7c3274f service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.536764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1199/2387] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:15.075596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:15.076985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:15.142675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a757-5713-4327-89d9-c4ab3ffe8afe service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:15.147168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:15.147853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:15.148133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:15.148409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] PATH_INFO: `/v3/users/c99b477597e447e98c9e7e9bfc9c19c3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:15.149621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99b477597e447e98c9e7e9bfc9c19c3)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:15.151569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:15.333041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99b477597e447e98c9e7e9bfc9c19c3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:15.333691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99b477597e447e98c9e7e9bfc9c19c3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:15.334295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99b477597e447e98c9e7e9bfc9c19c3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:15.341298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99b477597e447e98c9e7e9bfc9c19c3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:15.354168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99b477597e447e98c9e7e9bfc9c19c3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:15.439854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1189/2388] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:15 2022] DELETE /identity/v3/users/c99b477597e447e98c9e7e9bfc9c19c3 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:15.444240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:15.444739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:15.444842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:15.464029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:15.467070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:15.531852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1200/2389] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:15.541819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:15.543250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:15.601621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10dfc8-b6f4-4f69-bf52-a6314df5143c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:15.605636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:15.606316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:15.606673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:15.607726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:15.621608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:15.623192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:15.633039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1190/2390] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:16.061158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:16.062753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:16.081775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92364ac8-730c-4ac1-a839-69922cf0dd10 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:16.081775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:16.081775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:16.081775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:16.082179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] PATH_INFO: `/v3/projects/16dcc862ab844cd4a8a241eaf2bd8a91` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:16.082758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16dcc862ab844cd4a8a241eaf2bd8a91)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:16.084669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:16.206914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16dcc862ab844cd4a8a241eaf2bd8a91', 'request_id': 'req-211a5403-21a4-4978-81b3-67c6b7963df5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.212792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16dcc862ab844cd4a8a241eaf2bd8a91', 'request_id': 'req-211a5403-21a4-4978-81b3-67c6b7963df5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.219048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16dcc862ab844cd4a8a241eaf2bd8a91', 'request_id': 'req-211a5403-21a4-4978-81b3-67c6b7963df5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.304701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] The token cache is being invalidate because project 16dcc862ab844cd4a8a241eaf2bd8a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:16.304920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.305539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.307296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1201/2391] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:16 2022] DELETE /identity/v3/projects/16dcc862ab844cd4a8a241eaf2bd8a91 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:16.311104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:16.320899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab22703-a8bb-4121-9da4-94d6b58aea1b service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:16.338471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:16.377532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:16.378173 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:16.378549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:16.378744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] PATH_INFO: `/v3/users/0184bcac64a7422bb4b70b1bd3a56ca5` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:16.380261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0184bcac64a7422bb4b70b1bd3a56ca5)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:16.382106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:16.607131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0184bcac64a7422bb4b70b1bd3a56ca5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.607793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0184bcac64a7422bb4b70b1bd3a56ca5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.608404 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0184bcac64a7422bb4b70b1bd3a56ca5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.615381 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0184bcac64a7422bb4b70b1bd3a56ca5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.627848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0184bcac64a7422bb4b70b1bd3a56ca5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.688256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1191/2392] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:16 2022] DELETE /identity/v3/users/0184bcac64a7422bb4b70b1bd3a56ca5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:17.118527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:17.120197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:17.133810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a5403-21a4-4978-81b3-67c6b7963df5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:17.147489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:17.191910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:17.192525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.192806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.193076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] PATH_INFO: `/v3/projects/433e4be298f84dbbac46f8cf39bd6966` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.194166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=433e4be298f84dbbac46f8cf39bd6966)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:17.196199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:17.321592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '433e4be298f84dbbac46f8cf39bd6966', 'request_id': 'req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.327466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '433e4be298f84dbbac46f8cf39bd6966', 'request_id': 'req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.333732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '433e4be298f84dbbac46f8cf39bd6966', 'request_id': 'req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.476277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] The token cache is being invalidate because project 433e4be298f84dbbac46f8cf39bd6966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:17.476673 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.477431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.479538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1202/2393] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:17 2022] DELETE /identity/v3/projects/433e4be298f84dbbac46f8cf39bd6966 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:17.484237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:17.485663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:17.504836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bec774-ee1e-4a09-989c-8e1bd028969e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:17.560614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:17.561225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.561500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.561770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] PATH_INFO: `/v3/users/db58d7bc86174fba950a909002e70bd4` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.562892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db58d7bc86174fba950a909002e70bd4)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:17.564815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:17.740153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:17.742685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db58d7bc86174fba950a909002e70bd4'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.744620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db58d7bc86174fba950a909002e70bd4'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.744620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db58d7bc86174fba950a909002e70bd4'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.751328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db58d7bc86174fba950a909002e70bd4'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.762843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f0fc2660-0397-4c86-b9e4-7b9198bb02e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db58d7bc86174fba950a909002e70bd4'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.841122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1192/2394] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:17 2022] DELETE /identity/v3/users/db58d7bc86174fba950a909002e70bd4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:18.261196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.262694 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.276141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f52c-f009-4d7d-9d60-a589d2e70e6b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.333998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.334627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.334938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.335215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] PATH_INFO: `/v3/projects/dc3706438f764234a225b305db6fe569` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.336327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3706438f764234a225b305db6fe569)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.338244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.480418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3706438f764234a225b305db6fe569', 'request_id': 'req-d6bce71c-4e45-459b-840d-58df36628774'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.481881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:18.486683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3706438f764234a225b305db6fe569', 'request_id': 'req-d6bce71c-4e45-459b-840d-58df36628774'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.493716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3706438f764234a225b305db6fe569', 'request_id': 'req-d6bce71c-4e45-459b-840d-58df36628774'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.590343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] The token cache is being invalidate because project dc3706438f764234a225b305db6fe569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:18.590685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.591411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-d6bce71c-4e45-459b-840d-58df36628774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.592894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1203/2395] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:18 2022] DELETE /identity/v3/projects/dc3706438f764234a225b305db6fe569 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:18.597661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739cd3-110e-4045-b1bc-4e74fd681d82 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.598115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739cd3-110e-4045-b1bc-4e74fd681d82 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.598208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739cd3-110e-4045-b1bc-4e74fd681d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.618211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9e739cd3-110e-4045-b1bc-4e74fd681d82 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:18.619511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9e739cd3-110e-4045-b1bc-4e74fd681d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.683435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1193/2396] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:18.692296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f282-11cb-4214-9a31-149b7749dc60 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.692585 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f282-11cb-4214-9a31-149b7749dc60 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.692863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67f282-11cb-4214-9a31-149b7749dc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.713130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ab67f282-11cb-4214-9a31-149b7749dc60 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:18.714397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ab67f282-11cb-4214-9a31-149b7749dc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.723286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1204/2397] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:18.727504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e739cd3-110e-4045-b1bc-4e74fd681d82 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.737744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e739cd3-110e-4045-b1bc-4e74fd681d82 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.742637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.743260 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.743538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.743818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.744561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:18.744761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.746684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.759930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1194/2398] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:18.764813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67f282-11cb-4214-9a31-149b7749dc60 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.774094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67f282-11cb-4214-9a31-149b7749dc60 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.778627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.783393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.783393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.783393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.783393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.783393 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.863809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1205/2399] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:18 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:18.868469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.878510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d932f-8654-4c1d-a5f0-f68201ab8b09 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.937426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.938025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.938317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.938573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.939215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.941121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:19.029741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1195/2400] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:18 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:19.033745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:19.044949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f497b-93a0-4ceb-b513-b96ac47712b3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:19.049613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:19.050224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.050507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.050805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.051436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:19.053031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:19.062956 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1206/2401] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:19.066898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:19.077380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa69793-79f8-402a-a80a-bc222dbeef84 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:19.081929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eb7bd-4097-4fee-942f-7491062abfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:19.082512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eb7bd-4097-4fee-942f-7491062abfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.082825 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eb7bd-4097-4fee-942f-7491062abfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.083096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eb7bd-4097-4fee-942f-7491062abfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.083726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eb7bd-4097-4fee-942f-7491062abfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:19.085303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eb7bd-4097-4fee-942f-7491062abfe2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:19.096334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1196/2402] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:19.100232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:19.110215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30885a48-1ba7-45d7-917e-f1022c72b472 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:19.115177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:19.115799 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.116078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.116348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] PATH_INFO: `/v3/projects/947a3ce621a44edbbf2ea57ce77ac101/users/c442e73a4d32487c95fe4e77e58cdc28/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.132332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=947a3ce621a44edbbf2ea57ce77ac101, user_id=c442e73a4d32487c95fe4e77e58cdc28, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:19.135050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:19.197990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1207/2403] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:19 2022] PUT /identity/v3/projects/947a3ce621a44edbbf2ea57ce77ac101/users/c442e73a4d32487c95fe4e77e58cdc28/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:19.208748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c614b-de83-42de-a9cf-b4f543b280ba None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.211527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c614b-de83-42de-a9cf-b4f543b280ba None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.211527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c614b-de83-42de-a9cf-b4f543b280ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.237989 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-616c614b-de83-42de-a9cf-b4f543b280ba None None] MFA Rules not processed for user `c442e73a4d32487c95fe4e77e58cdc28`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:19.239599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-616c614b-de83-42de-a9cf-b4f543b280ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:19.253219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:19.309376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1197/2404] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:19.318952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:19.320874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:19.379147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93811-38d8-44bc-b387-578d1245bfcf admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:19.382789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:19.383391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.383744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.386401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.396125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:19.397565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:19.406962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1208/2405] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:22.221301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d73cb-9521-467f-8886-f256a3ed3fe6 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:22.221812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d73cb-9521-467f-8886-f256a3ed3fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:22.221920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d73cb-9521-467f-8886-f256a3ed3fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.241390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-135d73cb-9521-467f-8886-f256a3ed3fe6 None None] MFA Rules not processed for user `c442e73a4d32487c95fe4e77e58cdc28`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:22.242594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-135d73cb-9521-467f-8886-f256a3ed3fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:22.251198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1198/2406] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:25.187625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:25.189062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:25.252019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51434b-6f37-441c-9a95-32ce27fa37f7 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:25.255674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:25.256270 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:25.256550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:25.256822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:25.258566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:25.290823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:25.324929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:25.326505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:25.338416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1209/2407] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:32.235502 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=23 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:35.457239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=15 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:43.090220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e1b9b-e1a8-4848-ad99-1b54fd19d547 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.090759 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e1b9b-e1a8-4848-ad99-1b54fd19d547 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.090858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e1b9b-e1a8-4848-ad99-1b54fd19d547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.110933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c24e1b9b-e1a8-4848-ad99-1b54fd19d547 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:43.112160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c24e1b9b-e1a8-4848-ad99-1b54fd19d547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:43.171280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1199/2408] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:34:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:43.218978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:43.221988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:43.295524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98a22a-bd1e-43cc-abb9-cfbf311b4ce0 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:43.302599 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:43.303226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.303499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.303767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.312987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:43.314556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:43.322775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1210/2409] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:45.956276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1985bf-2360-40e7-bd4c-70977ea7316c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:45.956773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1985bf-2360-40e7-bd4c-70977ea7316c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:45.956978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1985bf-2360-40e7-bd4c-70977ea7316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:45.979066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5a1985bf-2360-40e7-bd4c-70977ea7316c None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:45.981190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5a1985bf-2360-40e7-bd4c-70977ea7316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:45.990579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1200/2410] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:34:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:46.005226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:46.016072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e1f10-0ebc-4d03-8a2c-1303b7d5932a service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:46.021552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:46.022412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:46.022715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:46.022988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:46.032682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:46.034323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:46.049800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1211/2411] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:47.580396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78743fc1-24de-4a28-a05d-a27c027cb5dc None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:47.581318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78743fc1-24de-4a28-a05d-a27c027cb5dc None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:47.581841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78743fc1-24de-4a28-a05d-a27c027cb5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:47.604635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-78743fc1-24de-4a28-a05d-a27c027cb5dc None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:47.605870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-78743fc1-24de-4a28-a05d-a27c027cb5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:47.613841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1201/2412] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 12:34:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:47.634031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:47.646470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0845a-7ed1-4766-94be-5386a0f3f897 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:47.652091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:47.653596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:47.653596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:47.653596 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:47.664094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:47.666177 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:47.675681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1212/2413] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:48.394431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297cb80-daa4-4050-98cb-7c6b6b3f70cc None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:48.394915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297cb80-daa4-4050-98cb-7c6b6b3f70cc None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:48.395266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297cb80-daa4-4050-98cb-7c6b6b3f70cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:48.441122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9297cb80-daa4-4050-98cb-7c6b6b3f70cc None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:48.442531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9297cb80-daa4-4050-98cb-7c6b6b3f70cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:48.456447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1202/2414] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:34:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:48.469707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:48.489860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d507d-d61e-4db6-8410-afddc198d6b8 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:48.496511 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:48.497416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:48.497869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:48.498340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:48.510540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:48.513698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:48.523011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1213/2415] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:49.166017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be25cd8-45f9-4746-bb85-0e877a0796b0 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:49.166532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be25cd8-45f9-4746-bb85-0e877a0796b0 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:49.167146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be25cd8-45f9-4746-bb85-0e877a0796b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:49.197244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-8be25cd8-45f9-4746-bb85-0e877a0796b0 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:49.198474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8be25cd8-45f9-4746-bb85-0e877a0796b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:49.207852 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1203/2416] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:49.228821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:49.245527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905acec-d62e-4f4d-81f6-32ea1de88006 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:49.256489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:49.263881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:49.263881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:49.263881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:49.268893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:49.271567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:49.282832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1214/2417] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:50.896117 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be25cd8-45f9-4746-bb85-0e877a0796b0 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:50.907539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be25cd8-45f9-4746-bb85-0e877a0796b0 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:50.912136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:50.912754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:50.913033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:50.913306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:50.915141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:51.055297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:51.056829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:51.072790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:51.074986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:51.076654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1204/2418] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:51.095538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207b478-a685-4ce9-8f87-b4ed1ad2e890 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:51.103154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ef42d-8ec7-4b1b-a609-83282dd96bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:51.104090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef42d-8ec7-4b1b-a609-83282dd96bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:51.104514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef42d-8ec7-4b1b-a609-83282dd96bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:51.105570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef42d-8ec7-4b1b-a609-83282dd96bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:51.116157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-179ef42d-8ec7-4b1b-a609-83282dd96bb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:51.174506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ef42d-8ec7-4b1b-a609-83282dd96bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:51.178466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ef42d-8ec7-4b1b-a609-83282dd96bb0 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:51.224784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1215/2419] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:52.990965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:53.119277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93e500-3ce0-4a62-acff-9b4495c77f39 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:53.128357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:53.129112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:53.129469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:53.129818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] PATH_INFO: `/v3/users/c442e73a4d32487c95fe4e77e58cdc28` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:53.131574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c442e73a4d32487c95fe4e77e58cdc28)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:53.135372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:53.146663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:53.791933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c442e73a4d32487c95fe4e77e58cdc28'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:53.791933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c442e73a4d32487c95fe4e77e58cdc28'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:53.791933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c442e73a4d32487c95fe4e77e58cdc28'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:53.798579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c442e73a4d32487c95fe4e77e58cdc28'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:53.817940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c442e73a4d32487c95fe4e77e58cdc28'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:53.917641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1205/2420] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:52 2022] DELETE /identity/v3/users/c442e73a4d32487c95fe4e77e58cdc28 => generated 0 bytes in 925 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:53.919156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:53.920425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:53.920771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:53.943652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:53.944984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.035079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1216/2421] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:54.045828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:54.047545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.120086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af70045-3784-4bb1-a01a-2fe205e8a30d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:54.129960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d6a6e-8a69-426a-9805-d72eab5973c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:54.130907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d6a6e-8a69-426a-9805-d72eab5973c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.131360 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d6a6e-8a69-426a-9805-d72eab5973c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.131799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d6a6e-8a69-426a-9805-d72eab5973c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:54.143308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d6a6e-8a69-426a-9805-d72eab5973c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:54.144972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d6a6e-8a69-426a-9805-d72eab5973c0 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:54.158829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1206/2422] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:54.498525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:54.501633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.520474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557499b-4328-4eff-bdc2-4b06a340936e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:54.527353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:54.528720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.528720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.528962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] PATH_INFO: `/v3/projects/947a3ce621a44edbbf2ea57ce77ac101` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:54.532808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=947a3ce621a44edbbf2ea57ce77ac101)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:54.536293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:54.558108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:54.763026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '947a3ce621a44edbbf2ea57ce77ac101', 'request_id': 'req-322d6ae1-8f12-411b-9eea-c4beae96afa7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.769328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '947a3ce621a44edbbf2ea57ce77ac101', 'request_id': 'req-322d6ae1-8f12-411b-9eea-c4beae96afa7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.776328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '947a3ce621a44edbbf2ea57ce77ac101', 'request_id': 'req-322d6ae1-8f12-411b-9eea-c4beae96afa7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.884280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] The token cache is being invalidate because project 947a3ce621a44edbbf2ea57ce77ac101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:54.884744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.885409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-322d6ae1-8f12-411b-9eea-c4beae96afa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.886983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1217/2423] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:54 2022] DELETE /identity/v3/projects/947a3ce621a44edbbf2ea57ce77ac101 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:54.892275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb17e-a1fb-409f-93f7-664578037996 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.892558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb17e-a1fb-409f-93f7-664578037996 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.892847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fb17e-a1fb-409f-93f7-664578037996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:54.912929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-6a0fb17e-a1fb-409f-93f7-664578037996 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:54.914214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6a0fb17e-a1fb-409f-93f7-664578037996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.975258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1207/2424] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:54.983079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e8f1a-58c3-49c2-9efd-dcbd5f6e7974 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.985817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e8f1a-58c3-49c2-9efd-dcbd5f6e7974 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.985817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e8f1a-58c3-49c2-9efd-dcbd5f6e7974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.004315 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-953e8f1a-58c3-49c2-9efd-dcbd5f6e7974 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:55.005480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-953e8f1a-58c3-49c2-9efd-dcbd5f6e7974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:55.013435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1218/2425] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:55.018110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fb17e-a1fb-409f-93f7-664578037996 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.027904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fb17e-a1fb-409f-93f7-664578037996 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.032972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.037498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.037498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.037498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.037498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:55.037498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.037498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.048700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1208/2426] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:55.053290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e8f1a-58c3-49c2-9efd-dcbd5f6e7974 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.062532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e8f1a-58c3-49c2-9efd-dcbd5f6e7974 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.067239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.067825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.068102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.068373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.068986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.070892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.149524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1219/2427] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:55.153761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.164477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df2db3-3224-4e55-9cf7-da8c336a46b3 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.173770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:55.220156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.220753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.221034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.221305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.221929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.223869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.310664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1209/2428] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:55.314771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.324923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c527ab-af8d-4cc7-aefc-8588ad50ded7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.329833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.330455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.330768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.331041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.331683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.333304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.343795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1220/2429] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:55.347638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.357848 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042cf98-83c1-4caa-a62f-3cb9ccc85494 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.363259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89caf02c-b478-4272-9cb2-da726135e85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.363849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caf02c-b478-4272-9cb2-da726135e85b admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.364127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caf02c-b478-4272-9cb2-da726135e85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.364702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caf02c-b478-4272-9cb2-da726135e85b admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.365278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89caf02c-b478-4272-9cb2-da726135e85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.366897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89caf02c-b478-4272-9cb2-da726135e85b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.377493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1210/2430] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:55.381581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.391420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff778efc-d123-4cfe-836a-d0382023ff57 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.395989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.396624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.396903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.397176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] PATH_INFO: `/v3/projects/bba3be3dcc1f48afbf2c60f233b8032d/users/b19f208c6c274e7d89a7bf08f908871d/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.412941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba3be3dcc1f48afbf2c60f233b8032d, user_id=b19f208c6c274e7d89a7bf08f908871d, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.416141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.519935 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1221/2431] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:55 2022] PUT /identity/v3/projects/bba3be3dcc1f48afbf2c60f233b8032d/users/b19f208c6c274e7d89a7bf08f908871d/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:55.525052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fb309-2135-4260-ab6c-f2d3ccac04df None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.525544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fb309-2135-4260-ab6c-f2d3ccac04df None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.525846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fb309-2135-4260-ab6c-f2d3ccac04df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.545655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-758fb309-2135-4260-ab6c-f2d3ccac04df None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:55.546999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-758fb309-2135-4260-ab6c-f2d3ccac04df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:55.606362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1211/2432] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:55.616370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.618049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:55.675116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66fbd-6901-44bf-ab4c-0a62a279ed23 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.678710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.679305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.679584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.679854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.688869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.690297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.699724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1222/2433] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:57.142753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb044a69-ec9b-409f-98b1-004b7d4139b9 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:57.143703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb044a69-ec9b-409f-98b1-004b7d4139b9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:57.143990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb044a69-ec9b-409f-98b1-004b7d4139b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:57.175771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-cb044a69-ec9b-409f-98b1-004b7d4139b9 None None] MFA Rules not processed for user `b19f208c6c274e7d89a7bf08f908871d`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:57.179370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cb044a69-ec9b-409f-98b1-004b7d4139b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:57.247628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1212/2434] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:57.262566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:57.262566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:57.313535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:57.318693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f144dc1-dce6-4797-8450-25e55148450c service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:57.322284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:57.322904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:57.326680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:57.326680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:57.332370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:57.333784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:57.342520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1223/2435] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:59.065012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb58c-b7d3-4686-b0d6-be45100f513a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:59.065531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb58c-b7d3-4686-b0d6-be45100f513a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:59.065647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb58c-b7d3-4686-b0d6-be45100f513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:59.087095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9c0fb58c-b7d3-4686-b0d6-be45100f513a None None] MFA Rules not processed for user `b19f208c6c274e7d89a7bf08f908871d`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:59.088333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9c0fb58c-b7d3-4686-b0d6-be45100f513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:59.096278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1213/2436] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:01.276583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:01.277855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:01.365238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e2cb-db31-49d5-a1c0-117d06caa8af service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:01.370745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe0d3c-8190-4b29-9146-9c98caafe939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:01.371351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe0d3c-8190-4b29-9146-9c98caafe939 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.371647 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe0d3c-8190-4b29-9146-9c98caafe939 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.371900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe0d3c-8190-4b29-9146-9c98caafe939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.373701 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5ebe0d3c-8190-4b29-9146-9c98caafe939 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:01.375493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ad4aa-818d-4c51-9663-63ca1965ea12 None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.376015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ad4aa-818d-4c51-9663-63ca1965ea12 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.376105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ad4aa-818d-4c51-9663-63ca1965ea12 None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.378715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1214/2437] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:35:01.383396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.383790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.384171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.407069 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:01.408667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:01.432881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1215/2438] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:01.445637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:01.447172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:01.456240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe0d3c-8190-4b29-9146-9c98caafe939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:01.458135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:01.458271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe0d3c-8190-4b29-9146-9c98caafe939 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:01.467806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1224/2439] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:01.528366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cf1e9-c304-4ca3-bf94-d99f7d811b66 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:01.535188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48071370-6fe9-406b-ad73-21e97eb8d355 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f33e1c573c044f086f99eb128c8f726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:01.537515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071370-6fe9-406b-ad73-21e97eb8d355 service glance] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.537515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071370-6fe9-406b-ad73-21e97eb8d355 service glance] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.537515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48071370-6fe9-406b-ad73-21e97eb8d355 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.548713 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48071370-6fe9-406b-ad73-21e97eb8d355 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:01.550264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48071370-6fe9-406b-ad73-21e97eb8d355 service glance] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:01.565890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1216/2440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:35:01.806462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064cbbf-1428-4d83-81be-6a003e3b0b79 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.806802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064cbbf-1428-4d83-81be-6a003e3b0b79 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.807084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064cbbf-1428-4d83-81be-6a003e3b0b79 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.808423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1225/2441] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:35:01.813362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4a10-f13b-422f-ac6e-e55c4da01f3c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.815510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4a10-f13b-422f-ac6e-e55c4da01f3c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.815510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4a10-f13b-422f-ac6e-e55c4da01f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.833415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3edd4a10-f13b-422f-ac6e-e55c4da01f3c None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:01.834621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3edd4a10-f13b-422f-ac6e-e55c4da01f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:01.841167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1217/2442] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:01.853517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064cbbf-1428-4d83-81be-6a003e3b0b79 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:01.855119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a064cbbf-1428-4d83-81be-6a003e3b0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:01.870572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064cbbf-1428-4d83-81be-6a003e3b0b79 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:01.874289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da214ec3-fe89-406a-8e38-ae700d306408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:01.874894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da214ec3-fe89-406a-8e38-ae700d306408 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.875179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da214ec3-fe89-406a-8e38-ae700d306408 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.875445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da214ec3-fe89-406a-8e38-ae700d306408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.882476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da214ec3-fe89-406a-8e38-ae700d306408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:01.883952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da214ec3-fe89-406a-8e38-ae700d306408 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:01.892674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1226/2443] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:35:02.011545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0c24-91fc-4260-ada7-7931ad56bfbf None None] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:02.011829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0c24-91fc-4260-ada7-7931ad56bfbf None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:02.012193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0c24-91fc-4260-ada7-7931ad56bfbf None None] PATH_INFO: `/` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:02.013608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1218/2444] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 12:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:35:02.020045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55537-56aa-4a6d-9331-968010274ce1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:02.020495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55537-56aa-4a6d-9331-968010274ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:02.022320 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55537-56aa-4a6d-9331-968010274ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:02.043780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a3c55537-56aa-4a6d-9331-968010274ce1 None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:02.044999 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a3c55537-56aa-4a6d-9331-968010274ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:02.054985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1227/2445] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:35:02.065636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e0c24-91fc-4260-ada7-7931ad56bfbf None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:02.076506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e0c24-91fc-4260-ada7-7931ad56bfbf None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:02.080276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:02.080915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:02.081194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:02.081550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:02.092148 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:02.094188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:02.104999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1219/2446] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:09.114925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06adde-1a07-4d00-9a14-1c310a6d6de8 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:09.115395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06adde-1a07-4d00-9a14-1c310a6d6de8 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:09.115488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06adde-1a07-4d00-9a14-1c310a6d6de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:09.134563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-be06adde-1a07-4d00-9a14-1c310a6d6de8 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:09.135766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-be06adde-1a07-4d00-9a14-1c310a6d6de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:09.189646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1228/2447] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 12:35:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:09.201323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:09.202720 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:09.271525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edf49e-ec25-47fa-a3da-b1883f379a71 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:09.276371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e07f4-e97e-4aaa-8a30-b518ed486d1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:09.276958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e07f4-e97e-4aaa-8a30-b518ed486d1b service swift] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:09.277234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e07f4-e97e-4aaa-8a30-b518ed486d1b service swift] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:09.277533 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e07f4-e97e-4aaa-8a30-b518ed486d1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:09.285495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e07f4-e97e-4aaa-8a30-b518ed486d1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:09.287095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e07f4-e97e-4aaa-8a30-b518ed486d1b service swift] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:09.295008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1220/2448] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:35:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:35:19.182751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:19.296589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=63 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:25.308238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06adde-1a07-4d00-9a14-1c310a6d6de8 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:25.317667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06adde-1a07-4d00-9a14-1c310a6d6de8 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:25.321396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:25.322045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:25.326233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:25.326233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:25.326233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:25.334682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:25.336193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:25.344679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1229/2449] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:35.340652 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:38.164631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec53043-c98c-4be1-ab1b-14017d44b270 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:38.165112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec53043-c98c-4be1-ab1b-14017d44b270 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:38.165296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec53043-c98c-4be1-ab1b-14017d44b270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:38.184855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-eec53043-c98c-4be1-ab1b-14017d44b270 None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:38.186049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-eec53043-c98c-4be1-ab1b-14017d44b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:38.193012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1221/2450] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 12:35:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:38.210675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:38.225018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a635eb9-94eb-495c-b75e-476675e983eb service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:38.230031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:38.230661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:38.230947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:38.231217 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:38.240606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:38.242161 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:38.252254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1230/2451] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:35:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:35:48.179307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:48.248775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:51.505264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec53043-c98c-4be1-ab1b-14017d44b270 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:51.517645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec53043-c98c-4be1-ab1b-14017d44b270 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:51.523522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b61d53-61c4-4a99-88ed-a28f65a98a29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:51.525826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b61d53-61c4-4a99-88ed-a28f65a98a29 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:51.525826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b61d53-61c4-4a99-88ed-a28f65a98a29 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:51.525826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b61d53-61c4-4a99-88ed-a28f65a98a29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:51.526272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-62b61d53-61c4-4a99-88ed-a28f65a98a29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:51.593950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b61d53-61c4-4a99-88ed-a28f65a98a29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:51.595576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b61d53-61c4-4a99-88ed-a28f65a98a29 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:51.604603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1222/2452] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:36:01.600494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=28 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:14.533658 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:36:14.545282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba45b3-3a65-48bf-ae2e-b0685cb8cf37 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:36:14.550605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412de87-c080-468d-b910-a6d776241559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:36:14.551248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412de87-c080-468d-b910-a6d776241559 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:14.551522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412de87-c080-468d-b910-a6d776241559 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:14.551791 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412de87-c080-468d-b910-a6d776241559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:14.554203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3412de87-c080-468d-b910-a6d776241559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:14.611314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412de87-c080-468d-b910-a6d776241559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:36:14.612813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412de87-c080-468d-b910-a6d776241559 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:36:14.622438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1231/2453] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:36:24.618306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=28 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:10.609616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a446cae-d507-45c1-8f97-dfa02d0456e5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:10.610130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a446cae-d507-45c1-8f97-dfa02d0456e5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:10.610241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a446cae-d507-45c1-8f97-dfa02d0456e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:10.628886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-7a446cae-d507-45c1-8f97-dfa02d0456e5 None None] MFA Rules not processed for user `b9252893998a490689497697663ca4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:10.630082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-7a446cae-d507-45c1-8f97-dfa02d0456e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:10.636792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1223/2454] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:37:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:37:17.118591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412de87-c080-468d-b910-a6d776241559 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:17.129840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412de87-c080-468d-b910-a6d776241559 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:17.136372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:17.136972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:17.137247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:17.137514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:17.139264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:17.152300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:17.153888 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:17.164338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1232/2455] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:20.623435 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:27.159993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:33.239768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a446cae-d507-45c1-8f97-dfa02d0456e5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:33.273342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a446cae-d507-45c1-8f97-dfa02d0456e5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:33.281833 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:33.282454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:33.282799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:33.283103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:33.285721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:33.303821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:33.307379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:33.319434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1224/2456] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:43.314030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:09.709439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:09.775219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe97d13-c594-472d-954c-b6900f7af0d9 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:09.780770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:09.781367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:09.781646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:09.781920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:09.783645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:09.850342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:09.851958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:09.861141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1233/2457] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:19.857043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=56 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:08.392816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:08.395354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:08.556267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0879dd7-0616-4e2b-a818-2b69f6c3a54f service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:08.561231 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:08.561824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:08.562098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:08.562366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:08.564096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:08.575092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:08.576640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:08.586709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1225/2458] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:18.582140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=63 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:58.092054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:58.110453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982973e-9d38-40e6-984f-8432e078df8f service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:58.113990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:58.114867 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:58.115146 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:58.115413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:58.123487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:58.124906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:58.133987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1234/2459] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:01.562949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:01.575728 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b28eb7-2182-4813-b848-ab64c9ef9075 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:01.581271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:01.581859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:01.582141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:01.582415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:01.590428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:01.592042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:01.601376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1226/2460] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:08.129653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:11.596864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:51.594645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:51.607471 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392af831-233c-4854-8705-ff19b89c7683 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:51.612860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:51.614385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:51.614668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:51.614952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:51.622793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:51.624431 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:51.652767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1235/2461] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:01.650194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=13 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:05.107785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:05.126350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad341c-65df-4ba8-8f9a-4e6c97fc4b1c service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:05.130590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:05.131190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:05.131462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:05.131724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:05.138819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:05.140223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:05.148641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1227/2462] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:15.133244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:15.134671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:15.145598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=10 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:15.159054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766694d8-d3b8-46cc-84eb-1edc32bdf176 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:15.163380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:15.163994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:15.164263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:15.164559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:15.172837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:15.174244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:15.201205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1236/2463] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:25.198472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=18 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:15.598002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:15.607899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb338b-856a-4328-b522-6f23326c6db7 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:15.611549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:15.612135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:15.612409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:15.612679 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:15.615276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:15.641346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:15.642829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:15.653546 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1228/2464] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:25.648857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=11 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:08.880846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:08.882328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:08.897887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10ad46-80db-4fb9-a258-7b2e0ff64991 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:08.903783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:08.904363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:08.904635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:08.904899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:08.912448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:08.914013 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:08.940487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1237/2465] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:18.938017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=13 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:44:10.373883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:44:10.375330 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:10.389253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cab2f-b65f-4f4b-a9e2-17db86339dc1 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:44:10.394122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:44:10.394754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:44:10.395033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:44:10.395312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:44:10.404013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:44:10.405587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:44:10.415485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1229/2466] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:44:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:44:20.411249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:44:58.679011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:44:58.680543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:58.735043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd4bf-fd5a-4d97-8411-3fa0eb3a258d service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:44:58.739893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:44:58.740480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:44:58.740753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:44:58.741019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:44:58.742843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:58.821813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:44:58.823283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:44:58.841861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1238/2467] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:44:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:02.222851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:02.232824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b062f-057a-4115-b379-e9b85f3b5926 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:02.236614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:02.237199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:02.237473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:02.237738 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:02.240568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:02.318471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:02.319909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:02.339159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1230/2468] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:08.839234 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=61 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:12.336526 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=41 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:52.002926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:52.011963 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbe113-bcc2-449b-b72f-a5c52a6d13a5 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:52.015527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:52.016104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:52.016377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:52.016644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:52.018405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:52.086566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:52.087944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:52.096971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1239/2469] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:46:02.092712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=33 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:46:09.063539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:46:09.065122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:46:09.131352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b069f-7033-46c5-a7d7-4fcac9937068 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:46:09.134806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:46:09.135391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:46:09.135671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:46:09.135938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:46:09.138307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:46:09.147377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:46:09.148788 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:46:09.156548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1231/2470] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:46:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:46:14.645642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:46:14.647292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:46:14.719581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c87dae-2872-4637-98c4-a4f030c150a3 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:46:14.724874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:46:14.725461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:46:14.725737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:46:14.726005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:46:14.727832 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:46:14.796715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:46:14.798325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:46:14.807539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1240/2471] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:46:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:46:19.153192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=31 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:46:24.803046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=61 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:48:09.356476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:48:09.366143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef732829-92e9-4245-9d85-d4e91bdba4ad service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:48:09.369756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:48:09.370948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:48:09.371235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:48:09.371510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:48:09.379798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:48:09.381206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:48:09.390055 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1232/2472] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:48:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:48:16.041349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:48:16.043087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:48:16.056070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481b59a-8710-42e2-bb71-41bbbd76671b service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:48:16.059812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:48:16.061997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:48:16.062286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:48:16.062560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:48:16.064468 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:48:16.087381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:48:16.089415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:48:16.117148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1241/2473] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:48:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:48:17.290634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:48:17.300490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c952a71-6133-4c45-90af-d2bd9bb95feb service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:48:17.304026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:48:17.304603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:48:17.304875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:48:17.305142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:48:17.315880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:48:17.317305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:48:17.328285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1233/2474] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:48:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:48:26.114384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=13 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:48:27.323548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=12 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:49:14.364895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:49:14.366504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:49:14.503009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7aafee-6729-4db6-bfb4-ce91e40ae094 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:49:14.508078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:49:14.508667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:49:14.508944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:49:14.509213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:49:14.511068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:49:14.522859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:49:14.524432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:49:14.534171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1242/2475] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:49:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:49:24.530063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=63 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:49:59.368749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:49:59.410410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fab92-df3a-4478-94b9-e15697a38859 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:49:59.415927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:49:59.416523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:49:59.420527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:49:59.420527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:49:59.425842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:49:59.427477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:49:59.436259 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1234/2476] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:49:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:50:03.175361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:50:03.187419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afad05-8aa3-4836-b26b-39a1dd8f5ef0 service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:50:03.193046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:50:03.193634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:50:03.193909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:50:03.194175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:50:03.202266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:50:03.203951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:50:03.212966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1243/2477] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:50:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:50:09.432279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=20 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:50:13.208774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:50:51.692162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:50:51.701297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ef0b9-10ef-41c4-93a8-e3f8785a7e86 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:50:51.705669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:50:51.706351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:50:51.706627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:50:51.706930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:50:51.715500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:50:51.716917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:50:51.744434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1235/2478] 10.222.0.45 () {60 vars in 1396 bytes} [Wed Aug 24 12:50:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:51:01.741772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=13 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:51:14.072874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:51:14.093900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4679b1-eb68-45df-b41f-9ee069f30f6e service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:51:14.097622 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:51:14.098265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:51:14.098538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:51:14.098844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:51:14.107944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:51:14.109825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:51:14.120879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1244/2479] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:51:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:51:14.654232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:51:14.673219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b25f-acc1-4a0b-9bf5-1812713e250a service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:51:14.676906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:51:14.677494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:51:14.677769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:51:14.678039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:51:14.686580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:51:14.688029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:51:14.697955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1236/2480] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:51:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:51:24.117264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:51:24.693485 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=10 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:53:09.239573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:53:09.249537 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2676-f136-40db-848a-07559cc08b66 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:53:09.253572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:53:09.254157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:53:09.255109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:53:09.255384 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:53:09.258103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:53:09.339847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:53:09.341297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:53:09.359506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1245/2481] 10.222.0.45 () {60 vars in 1396 bytes} [Wed Aug 24 12:53:09 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:53:16.497094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:53:16.506916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eeee02-6b24-470a-967a-04814cadf630 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:53:16.510416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:53:16.511034 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:53:16.511312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:53:16.511581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:53:16.521461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:53:16.522901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:53:16.533553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1237/2482] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:53:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:53:17.487870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:53:17.497106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2373a2-901f-47c8-9ac2-a78c509e9010 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:53:17.500827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:53:17.501414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:53:17.501691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:53:17.501959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:53:17.503835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:53:17.529686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:53:17.531132 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:53:17.542197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1246/2483] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:53:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:53:26.528795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:53:27.537353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=52 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:54:17.435332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:54:17.437448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:54:17.452143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21899-1ca4-414e-882c-39a8d99d7e4b service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:54:17.457219 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:54:17.457836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:54:17.458114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:54:17.458389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:54:17.466883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:54:17.468540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:54:17.478559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1238/2484] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:54:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:54:19.760388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:54:19.772208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46eb13-0b2f-409e-970d-a1beaafdf6e9 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:54:19.777971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:54:19.778578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:54:19.778903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:54:19.779421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:54:19.781277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:54:19.794760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:54:19.796368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:54:19.808349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1247/2485] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:54:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:54:27.474274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:54:29.803546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:55:01.732714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:01.747347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4723717-27b7-41f4-9cda-f0b7068ef873 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:01.752991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3bd0c-7f96-4880-8752-443ded05fbc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:01.753592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3bd0c-7f96-4880-8752-443ded05fbc8 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:01.754482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3bd0c-7f96-4880-8752-443ded05fbc8 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:01.754808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3bd0c-7f96-4880-8752-443ded05fbc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:01.756844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c8f3bd0c-7f96-4880-8752-443ded05fbc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:01.769939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3bd0c-7f96-4880-8752-443ded05fbc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:01.771986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3bd0c-7f96-4880-8752-443ded05fbc8 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:01.786361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1239/2486] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:55:02.715098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:02.725567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607dc12-74fa-45c3-b937-43d9ea3f7941 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:02.729851 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:02.730460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:02.730886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:02.731162 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:02.733045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:02.840534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:02.841980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:02.862287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1248/2487] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:55:04.749908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b62d47-614e-49ae-b0f8-61e75b25eca0 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:04.750455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b62d47-614e-49ae-b0f8-61e75b25eca0 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:04.750584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b62d47-614e-49ae-b0f8-61e75b25eca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:04.782566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a1b62d47-614e-49ae-b0f8-61e75b25eca0 None None] MFA Rules not processed for user `6b94f244190e4af089d070ad83806c38`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:55:04.783821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a1b62d47-614e-49ae-b0f8-61e75b25eca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:04.791297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1240/2488] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 12:55:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:55:05.123320 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:05.131754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe495d-283f-4f8a-b988-4b3ec305978c service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:05.135726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:05.136321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:05.136597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:05.136863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:05.138559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:05.148119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:05.150083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:05.158067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1249/2489] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:55:05.610815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b62d47-614e-49ae-b0f8-61e75b25eca0 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:05.627213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b62d47-614e-49ae-b0f8-61e75b25eca0 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:05.635067 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:05.635866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:05.636192 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:05.636472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:05.639570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:05.654734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:05.666835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:05.667891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1241/2490] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:55:06.601543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:06.601543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:06.670554 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4abf0-298c-4aef-baf1-4ad0d5c78ecd service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:06.674455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e1905-052b-4b47-a8e8-86bc8fa16028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:06.675095 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e1905-052b-4b47-a8e8-86bc8fa16028 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:06.675372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e1905-052b-4b47-a8e8-86bc8fa16028 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:06.675723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e1905-052b-4b47-a8e8-86bc8fa16028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:06.677496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-621e1905-052b-4b47-a8e8-86bc8fa16028 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:06.747472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=100 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:55:06.765465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e1905-052b-4b47-a8e8-86bc8fa16028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:06.767071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e1905-052b-4b47-a8e8-86bc8fa16028 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:06.787549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1250/2491] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:55:15.663587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=18 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:55:16.708661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:16.721993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9da9c-c160-4883-a526-c2870aa13b29 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:16.727462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:16.728689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:16.728977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:16.729249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:16.731042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:16.743828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:16.745414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:16.756162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1242/2492] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:55:16.837010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=13 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:55:26.751585 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:55:35.209427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0d3b-ac6b-4173-b17e-7638755be5cf None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:35.209861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0d3b-ac6b-4173-b17e-7638755be5cf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:35.209960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0d3b-ac6b-4173-b17e-7638755be5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:35.270702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-0d0f0d3b-ac6b-4173-b17e-7638755be5cf None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:55:35.272367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0d0f0d3b-ac6b-4173-b17e-7638755be5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:35.335035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1251/2493] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 12:55:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:55:35.469293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:35.470930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:35.558592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748ac7-2386-405f-8b9c-f8f7aaf49346 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:35.563544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:35.564136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:35.564417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:35.564692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:35.573086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:35.574658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:35.582496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1243/2494] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 12:55:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:55:36.816312 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0d3b-ac6b-4173-b17e-7638755be5cf None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:36.826078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0d3b-ac6b-4173-b17e-7638755be5cf None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:36.829668 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:36.830301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:36.830577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:36.830908 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:36.832677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:55:36.921918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:36.923395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:36.941960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1252/2495] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:55:45.579441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=41 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:55:46.939392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=87 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:55:51.162415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:55:51.174250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e179e-400f-4a04-b2c9-deadc234a19f service swift] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:55:51.179652 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:55:51.183949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:55:51.183949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:55:51.183949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:55:51.188724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:55:51.190289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:55:51.199774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1244/2496] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:55:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:56:01.195692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:56:14.048284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:56:14.050063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:56:14.120570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014a2e1-adb9-4c59-8f43-6dae87e23f61 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:56:14.125563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:56:14.126435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:56:14.126900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:56:14.127325 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:56:14.130843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:56:14.144857 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:56:14.147346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:56:14.158871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1253/2497] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:56:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:56:14.508498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:56:14.521480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5aae78-a7de-44a6-aa54-622746dba7db service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:56:14.527778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:56:14.528744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:56:14.529250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:56:14.529794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:56:14.541477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:56:14.543109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:56:14.573746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1245/2498] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:56:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:56:24.155510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=31 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:56:24.570932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=13 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:58:16.963489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:58:16.965123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:58:16.975756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b179919-d678-4c96-b98e-ecd330821056 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:58:16.979345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:58:16.979930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:58:16.980208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:58:16.980479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:58:16.982229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:58:17.058444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:58:17.060024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:58:17.080570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1254/2499] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:58:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:58:18.186910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:58:18.196300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd5a39-99ca-4f1c-aeb2-8e1110237723 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:58:18.199869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:58:18.200469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:58:18.200746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:58:18.201016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:58:18.211132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:58:18.212588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:58:18.223731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1246/2500] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:58:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:58:27.077915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=41 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:58:28.219007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:59:24.390093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:59:24.391659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:59:24.499579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33163894-f3a3-49ab-8e3e-f3a29f283b2a service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:59:24.504549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07277290-da3e-429a-8670-c94f2f9bfe4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:59:24.505136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07277290-da3e-429a-8670-c94f2f9bfe4f service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:59:24.505411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07277290-da3e-429a-8670-c94f2f9bfe4f service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:59:24.505678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07277290-da3e-429a-8670-c94f2f9bfe4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:59:24.507503 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-07277290-da3e-429a-8670-c94f2f9bfe4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:59:24.519081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07277290-da3e-429a-8670-c94f2f9bfe4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:59:24.520636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07277290-da3e-429a-8670-c94f2f9bfe4f service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:59:24.531438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1255/2501] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 12:59:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:59:34.526100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=49 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:00:20.219013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:00:20.220589 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:00:20.243981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b266b9-d959-4b4c-bf1b-6db147024451 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:00:20.248035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:00:20.248709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:00:20.248990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:00:20.249900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:00:20.274365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:00:20.275839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:00:20.285494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1247/2502] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:00:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:00:20.777614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b7d1-a585-433f-a80d-0de54492a3ab None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:00:20.778051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b7d1-a585-433f-a80d-0de54492a3ab None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:00:20.778186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b7d1-a585-433f-a80d-0de54492a3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:00:20.817394 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b279b7d1-a585-433f-a80d-0de54492a3ab None None] MFA Rules not processed for user `751f05f1ff6c4593870073d8f47e0db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:00:20.818597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b279b7d1-a585-433f-a80d-0de54492a3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:00:20.872116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1256/2503] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:00:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:00:21.184753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:00:21.194542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b9201-cd37-486c-a80d-c6374132ed71 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:00:21.198087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:00:21.198710 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:00:21.198991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:00:21.199243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:00:21.200960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:00:21.211256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:00:21.221698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:00:21.221698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1248/2504] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:00:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:00:21.641508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b7d1-a585-433f-a80d-0de54492a3ab None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:00:21.661239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b7d1-a585-433f-a80d-0de54492a3ab None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:00:21.666235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:00:21.666929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:00:21.667274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:00:21.667633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:00:21.677644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:00:21.679290 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:00:21.690199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1257/2505] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:00:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:00:25.609736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:00:25.612027 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:00:25.628229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4650c8-c0e1-43dc-a6a1-00b4fdad472d service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:00:25.633909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:00:25.634567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:00:25.634867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:00:25.635141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:00:25.645074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:00:25.646864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:00:25.658298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1249/2506] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:00:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:00:31.685385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=42 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:00:32.281800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:00:32.293639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a779e-998a-480b-9e7e-718ad370798d service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:00:32.299541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:00:32.300128 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:00:32.300581 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:00:32.300881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:00:32.309900 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:00:32.313772 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:00:32.322492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1258/2507] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:00:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:00:35.653489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=29 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:00:42.318037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:01:07.953868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:01:07.963512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c5a41-b8f9-40ad-b405-af3533e9bf61 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:01:07.966996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:01:07.967569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:01:07.967843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:01:07.968106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:01:07.969899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:01:08.046035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:01:08.047467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:01:08.065444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1250/2508] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:01:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:01:08.112105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:01:08.121712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378163-ce29-46ae-90d3-6658280cf758 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:01:08.125251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:01:08.125827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:01:08.126102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:01:08.126387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:01:08.135065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:01:08.136467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:01:08.145088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1259/2509] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:01:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:01:08.171541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:01:08.180124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5ccc6-5ef5-43f9-9652-94e343411914 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:01:08.183659 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:01:08.184237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:01:08.184510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:01:08.184777 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:01:08.193500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:01:08.195002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:01:08.203835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1251/2510] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:01:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:01:18.141184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:01:18.199653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=47 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:01:21.027258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:01:21.046269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08f3af-3b11-4c28-b7d1-b9ca46cc5326 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:01:21.049803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:01:21.050382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:01:21.050680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:01:21.050955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:01:21.057953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:01:21.059383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:01:21.067766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1260/2511] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:01:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:01:31.063801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:01:41.401933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:01:41.414808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0559d3-b63c-4a67-adb6-7e172725517c service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:01:41.420261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:01:41.420857 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:01:41.421142 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:01:41.421413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:01:41.423307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:01:41.434671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:01:41.436263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:01:41.445245 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1252/2512] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:01:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:01:51.441032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:04:28.339852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:04:28.352469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95982c19-dea5-416b-ab8f-7e412b1964d6 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:04:28.357387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:04:28.358015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:04:28.358293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:04:28.358567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:04:28.367342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:04:28.368902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:04:28.378553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1261/2513] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:04:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:04:38.374482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:06:20.460425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:06:20.523167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64987a6b-9c35-4350-a9a0-09222f2820b2 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:06:20.526853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:06:20.527449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:06:20.527731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:06:20.528004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:06:20.529849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:06:20.607374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:06:20.608736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:06:20.627030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1253/2514] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:06:20.674048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:06:20.683795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df32df9-4bd1-48aa-9eae-86458188b48c service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:06:20.687328 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:06:20.687986 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:06:20.688263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:06:20.688531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:06:20.697137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:06:20.698539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:06:20.707686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1262/2515] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:06:20.756075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:06:20.765199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510756a0-0f76-4fd2-a36e-893e96b4c58c service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:06:20.768936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:06:20.769522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:06:20.769818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:06:20.770087 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:06:20.771906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:06:20.848988 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=100 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:06:20.851313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:06:20.852773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:06:20.872195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1254/2516] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:06:20.914967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:06:20.923796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9853-d357-4098-8ddc-e8435a94f714 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:06:20.927326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:06:20.927904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:06:20.928179 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:06:20.928444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:06:20.930236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:06:21.009444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:06:21.010909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:06:21.029745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1263/2517] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:06:24.084724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:06:24.086324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:06:24.153051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec7e89-b296-4e1c-9675-5358fdfd34af service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:06:24.157574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:06:24.158179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:06:24.158455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:06:24.158750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:06:24.160573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:06:24.171339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:06:24.172840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:06:24.180863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1255/2518] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:06:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:06:31.026911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=47 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:06:34.177363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=37 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:10.804201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:10.806260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:10.829712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8cb23-7232-4a4d-b342-b602c079a29c service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:10.833289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:10.833940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:10.834215 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:10.834480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:10.843520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:10.844933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:10.873683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1264/2519] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:19.401880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:19.417507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc246b-781c-4676-afed-6d09c2090d97 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:19.421930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:19.422883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:19.423436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:19.423922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:19.429929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:19.456721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:19.458193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:19.470232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1256/2520] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:20.681399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:20.690140 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbda2-2b38-40af-9d17-40b0ee5177e5 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:20.693876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:20.697928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:20.697928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:20.697928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:20.705435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:20.707568 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:20.718006 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1265/2521] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:07:21.466381 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:21.475078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e275b-8fe9-449a-a080-2874dcb90196 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:21.479249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:21.479876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:21.480155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:21.480429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:21.482271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:21.565322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:21.566778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:21.585890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1257/2522] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:23.275707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:23.290750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194ce22-1bab-458f-977a-ea914c4e54dd service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:23.305789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:23.306422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:23.306738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:23.307158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:23.312155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:23.352359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:23.355249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:23.368661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1266/2523] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:23.996466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:24.009214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2aa5e-86b5-4fc1-9e4f-dc299bb97be7 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:24.014600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:24.015247 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:24.015522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:24.015792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:24.017587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:24.035380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:24.039458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:24.050001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1258/2524] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:29.102358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:29.112174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd99a9-03e8-41b6-a8d2-9dad68e358a7 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:29.115798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e423f32-3567-40b4-81e9-42452bd51a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:29.116386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e423f32-3567-40b4-81e9-42452bd51a16 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:29.116793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e423f32-3567-40b4-81e9-42452bd51a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:29.117113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e423f32-3567-40b4-81e9-42452bd51a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:29.118925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3e423f32-3567-40b4-81e9-42452bd51a16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:29.130617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e423f32-3567-40b4-81e9-42452bd51a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:29.132519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e423f32-3567-40b4-81e9-42452bd51a16 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:29.142000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1267/2525] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:30.693285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:30.702696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74dd8a-68e3-45ed-8a27-24e7e880c220 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:30.706401 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd854f-115a-42be-bde1-47b57fc00f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:30.707031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd854f-115a-42be-bde1-47b57fc00f6e service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:30.707313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd854f-115a-42be-bde1-47b57fc00f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:30.707584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd854f-115a-42be-bde1-47b57fc00f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:30.709278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4fdd854f-115a-42be-bde1-47b57fc00f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:30.721569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd854f-115a-42be-bde1-47b57fc00f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:30.724004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd854f-115a-42be-bde1-47b57fc00f6e service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:30.753851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1259/2526] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:33.706448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff1147-05bd-4d9a-8d01-192998d55609 None None] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:33.707036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff1147-05bd-4d9a-8d01-192998d55609 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:33.707369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff1147-05bd-4d9a-8d01-192998d55609 None None] PATH_INFO: `/` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:33.709197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1268/2527] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 13:07:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:07:33.720940 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:33.722236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:33.722236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:33.790476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:33.791635 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:33.801273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1260/2528] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 13:07:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:33.813287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff1147-05bd-4d9a-8d01-192998d55609 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:33.821514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=37 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:33.829021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff1147-05bd-4d9a-8d01-192998d55609 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:33.835121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:33.835808 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:33.839886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:33.839886 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:33.845559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:33.847233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:33.859877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1269/2529] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:33.894304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:33.895982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:33.925273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad73a4-87b2-4767-aca9-c5d1c4a8be34 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:33.933745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:33.934858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:33.935322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:33.935828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] PATH_INFO: `/v3/users/6b94f244190e4af089d070ad83806c38` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:33.937683 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b94f244190e4af089d070ad83806c38)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:33.942098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:34.093802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:34.106490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b94f244190e4af089d070ad83806c38'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.107266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b94f244190e4af089d070ad83806c38'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.107943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b94f244190e4af089d070ad83806c38'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.119197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b94f244190e4af089d070ad83806c38'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.131328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c7da15a9-f684-4b9a-8a30-5549436a6767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b94f244190e4af089d070ad83806c38'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.204162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1261/2530] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:33 2022] DELETE /identity/v3/users/6b94f244190e4af089d070ad83806c38 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:34.463444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:34.474229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e2b9-ae1e-4cc0-ad09-3e41a490facf service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:34.533531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:34.534158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:34.534438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:34.534740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] PATH_INFO: `/v3/projects/504b03560cf34426a08958b471c91646` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:34.535869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=504b03560cf34426a08958b471c91646)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:34.537822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:34.635497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504b03560cf34426a08958b471c91646', 'request_id': 'req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.641022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504b03560cf34426a08958b471c91646', 'request_id': 'req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.648021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504b03560cf34426a08958b471c91646', 'request_id': 'req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.736398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] The token cache is being invalidate because project 504b03560cf34426a08958b471c91646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:07:34.736966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.737532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b9dbee8-8172-482d-8620-b81fe4d5e2dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:34.739210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1270/2531] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:34 2022] DELETE /identity/v3/projects/504b03560cf34426a08958b471c91646 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:34.744685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb71a18-6212-45ba-bb5f-43c4946fb924 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:34.745180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb71a18-6212-45ba-bb5f-43c4946fb924 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:34.745270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb71a18-6212-45ba-bb5f-43c4946fb924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:34.794307 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-6cb71a18-6212-45ba-bb5f-43c4946fb924 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:34.795551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6cb71a18-6212-45ba-bb5f-43c4946fb924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:34.865053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1262/2532] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:34.872932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00c5e5-8950-4e1b-94c1-111d987ce74e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:34.879038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00c5e5-8950-4e1b-94c1-111d987ce74e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:34.879038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00c5e5-8950-4e1b-94c1-111d987ce74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:34.895050 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3e00c5e5-8950-4e1b-94c1-111d987ce74e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:34.896563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3e00c5e5-8950-4e1b-94c1-111d987ce74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:34.907311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1271/2533] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:34.911429 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb71a18-6212-45ba-bb5f-43c4946fb924 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:34.930319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb71a18-6212-45ba-bb5f-43c4946fb924 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:34.935271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:34.935855 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:34.936336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:34.936811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:34.937610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:07:34.938096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:34.940177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:34.954741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1263/2534] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:07:34.959291 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00c5e5-8950-4e1b-94c1-111d987ce74e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:34.972025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e00c5e5-8950-4e1b-94c1-111d987ce74e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:34.982173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:34.983441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:34.983796 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:34.984408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:34.995102 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:34.997298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:35.124544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1272/2535] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:34 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:35.128724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:35.144376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6891fe5-84f4-41b1-bc3d-6ebaab54a317 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:35.215042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:35.216024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:35.216306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:35.216577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:35.217208 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:35.219134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:35.337822 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1264/2536] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:35 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:35.341797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:35.352011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42cf11-ac9f-4ebf-a612-a58fd50414c2 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:35.356516 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:35.357099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:35.357371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:35.357645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:35.358273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:35.360213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:35.370504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1273/2537] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:35.374263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:35.380717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:35.385064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5016e5-0e05-49d8-a810-e0c9ebe72111 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:35.389717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e666c-6b4a-44f3-ae43-d31dfd6d8425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:35.390317 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e666c-6b4a-44f3-ae43-d31dfd6d8425 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:35.392787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e666c-6b4a-44f3-ae43-d31dfd6d8425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:35.392787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e666c-6b4a-44f3-ae43-d31dfd6d8425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:35.392787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e666c-6b4a-44f3-ae43-d31dfd6d8425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:35.393170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e666c-6b4a-44f3-ae43-d31dfd6d8425 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:35.408037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1265/2538] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:35.409134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:35.423603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8bb64-17f6-4dda-b84e-9e67f2c5b396 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:35.431685 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:35.432340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:35.432618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:35.432889 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] PATH_INFO: `/v3/projects/bcfccce80ab74e6591df6ef220a26a95/users/256b4ce317664b5493e81ab3a5a0f3bf/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:35.450924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcfccce80ab74e6591df6ef220a26a95, user_id=256b4ce317664b5493e81ab3a5a0f3bf, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:35.453567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:35.532402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1274/2539] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:35 2022] PUT /identity/v3/projects/bcfccce80ab74e6591df6ef220a26a95/users/256b4ce317664b5493e81ab3a5a0f3bf/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:35.542471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:35.542821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:35.543095 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:35.572616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] MFA Rules not processed for user `256b4ce317664b5493e81ab3a5a0f3bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:35.573860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:35.638486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1266/2540] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:35.649593 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:35.651293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:35.661571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:35.724519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f7ae4-1a36-45ab-ba34-bfe74156a59e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:35.730858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:35.731493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:35.731774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:35.732043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:35.744190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:35.744190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:35.745212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:35.746634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:35.753142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1275/2541] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:35.816899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684997df-d3a3-42cf-aa34-d85a0028f4b9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:35.821753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:35.822417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:35.822749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:35.823022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] PATH_INFO: `/v3/users/b9252893998a490689497697663ca4d2` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:35.824171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9252893998a490689497697663ca4d2)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:35.826066 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:36.025907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9252893998a490689497697663ca4d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.026573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9252893998a490689497697663ca4d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.027301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9252893998a490689497697663ca4d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.034180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9252893998a490689497697663ca4d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.046318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-47898266-3aa6-4c5f-be46-a2d83c741e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9252893998a490689497697663ca4d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.142992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1267/2542] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:35 2022] DELETE /identity/v3/users/b9252893998a490689497697663ca4d2 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:07:36.366019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:36.378083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28578ca3-58cb-4575-8b2f-5081995c47d1 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:36.451762 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:36.452459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:36.452836 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:36.453117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] PATH_INFO: `/v3/projects/99f0beb902e4472eae20f0e6c8dc3340` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:36.454245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f0beb902e4472eae20f0e6c8dc3340)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:36.456181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:36.583149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f0beb902e4472eae20f0e6c8dc3340', 'request_id': 'req-f772cf60-96b4-463c-81e2-dcfcd27d941b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.589410 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f0beb902e4472eae20f0e6c8dc3340', 'request_id': 'req-f772cf60-96b4-463c-81e2-dcfcd27d941b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.595847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f0beb902e4472eae20f0e6c8dc3340', 'request_id': 'req-f772cf60-96b4-463c-81e2-dcfcd27d941b'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.680223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] The token cache is being invalidate because project 99f0beb902e4472eae20f0e6c8dc3340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:07:36.680442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.681070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f772cf60-96b4-463c-81e2-dcfcd27d941b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:36.682582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1276/2543] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:36 2022] DELETE /identity/v3/projects/99f0beb902e4472eae20f0e6c8dc3340 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:36.688430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e04d1-72ba-4232-b3f9-6237c276a3da None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:36.688749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e04d1-72ba-4232-b3f9-6237c276a3da None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:36.689023 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e04d1-72ba-4232-b3f9-6237c276a3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:36.696701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:36.708927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f83e04d1-72ba-4232-b3f9-6237c276a3da None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:36.710665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f83e04d1-72ba-4232-b3f9-6237c276a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:36.768976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1268/2544] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:36.778202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066a7d-b3d2-4a6c-a321-8725283bfc40 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:36.778624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066a7d-b3d2-4a6c-a321-8725283bfc40 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:36.778784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066a7d-b3d2-4a6c-a321-8725283bfc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:36.798618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-fd066a7d-b3d2-4a6c-a321-8725283bfc40 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:36.799846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-fd066a7d-b3d2-4a6c-a321-8725283bfc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:36.807723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1277/2545] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:36.811994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e04d1-72ba-4232-b3f9-6237c276a3da None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:36.822011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e04d1-72ba-4232-b3f9-6237c276a3da None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:36.826682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:36.827293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:36.827576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:36.827851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:36.828499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:07:36.828800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:36.830398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:36.843044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1269/2546] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:07:36.847551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066a7d-b3d2-4a6c-a321-8725283bfc40 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:36.858639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066a7d-b3d2-4a6c-a321-8725283bfc40 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:36.863930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:36.864529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:36.864805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:36.865072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:36.865704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:36.867667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:36.883562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:36.956014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1278/2547] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:36 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:36.959921 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:36.970530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95387f5-0d19-4c69-975c-6549c92a3524 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.025689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.026277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.026554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.026856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.027473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.029362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:37.117500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1270/2548] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:36 2022] POST /identity/v3/users => generated 301 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:37.121824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:37.132679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aa6b1-4803-4e8b-a923-e3ebc3d8b0ce admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.138118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.138741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.139020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.139289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.139924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.141547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:37.152736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1279/2549] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:37.156695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:37.166906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec9363-984f-4a66-9419-dd46356fc018 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.171554 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6eeef-e9ef-4d9b-9ade-ef8876a27b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.172205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6eeef-e9ef-4d9b-9ade-ef8876a27b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.172549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6eeef-e9ef-4d9b-9ade-ef8876a27b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.172885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6eeef-e9ef-4d9b-9ade-ef8876a27b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.173520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6eeef-e9ef-4d9b-9ade-ef8876a27b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.175124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6eeef-e9ef-4d9b-9ade-ef8876a27b74 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:37.185419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1271/2550] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:37.189576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:37.199985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df36782-946c-4d98-a3e9-fb63819cfc03 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.204583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.205212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.205494 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.205765 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] PATH_INFO: `/v3/projects/3cf3d00bd3ff4a00b370d9ee91405884/users/ff56d19d3761424dafc57a905880e0f9/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.340370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf3d00bd3ff4a00b370d9ee91405884, user_id=ff56d19d3761424dafc57a905880e0f9, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.343030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:37.435600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1280/2551] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:37 2022] PUT /identity/v3/projects/3cf3d00bd3ff4a00b370d9ee91405884/users/ff56d19d3761424dafc57a905880e0f9/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:37.445682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daf723-2892-4383-905c-12a46fe5c7ce None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.445963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daf723-2892-4383-905c-12a46fe5c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.446230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daf723-2892-4383-905c-12a46fe5c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.474537 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a6daf723-2892-4383-905c-12a46fe5c7ce None None] MFA Rules not processed for user `ff56d19d3761424dafc57a905880e0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:37.476548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a6daf723-2892-4383-905c-12a46fe5c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:37.540387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:37.546239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1272/2552] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:37.550949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:37.624246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdd621-6e94-46a2-aaf1-7732d7fbbcae admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.626401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.627015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.627287 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.627551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.628156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.630033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:37.702604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1281/2553] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:37 2022] POST /identity/v3/projects => generated 399 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:37.706804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daf723-2892-4383-905c-12a46fe5c7ce None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:37.717386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daf723-2892-4383-905c-12a46fe5c7ce None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.776162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.776816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.777044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.777316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.777943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.779935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:37.859837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1273/2554] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:37 2022] POST /identity/v3/users => generated 306 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:37.864092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:37.875403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfcc1a-776d-45de-8428-432fd08b5197 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.880330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.885400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.885400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.885400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.885400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.885400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:37.895199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1282/2555] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:07:37.900022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:37.911543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d8080-530b-45f1-ab4f-2ca01535eb3a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:37.916917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:37.917619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:37.917902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:37.918177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] PATH_INFO: `/v3/projects/24682603ccd5412fa9c7a27011bce0a3/users/e852eb3e1f574f8a991dc72089a2af36/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:37.943439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24682603ccd5412fa9c7a27011bce0a3, user_id=e852eb3e1f574f8a991dc72089a2af36, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:37.948663 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:38.027506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1274/2556] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:37 2022] PUT /identity/v3/projects/24682603ccd5412fa9c7a27011bce0a3/users/e852eb3e1f574f8a991dc72089a2af36/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:38.032876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:38.047190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc22c73-eda4-4ba0-8cfc-132b5ab91884 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:38.113376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd190dd-3b01-4442-bef9-be6e4dfdc606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:38.113980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd190dd-3b01-4442-bef9-be6e4dfdc606 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:38.114263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd190dd-3b01-4442-bef9-be6e4dfdc606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:38.114536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd190dd-3b01-4442-bef9-be6e4dfdc606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:38.115211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd190dd-3b01-4442-bef9-be6e4dfdc606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:38.117508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd190dd-3b01-4442-bef9-be6e4dfdc606 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:38.128924 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1283/2557] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:38.131489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:38.142037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb5d34-c55c-4808-9a4e-4f93f2490507 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:38.147106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:38.147815 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:38.148092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:38.148367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] PATH_INFO: `/v3/projects/24682603ccd5412fa9c7a27011bce0a3/users/e852eb3e1f574f8a991dc72089a2af36/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:38.150341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24682603ccd5412fa9c7a27011bce0a3, user_id=e852eb3e1f574f8a991dc72089a2af36, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:38.153028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:38.164706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1275/2558] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:38 2022] PUT /identity/v3/projects/24682603ccd5412fa9c7a27011bce0a3/users/e852eb3e1f574f8a991dc72089a2af36/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:38.174871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099449-2202-4d4d-932a-daed14dda6d6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:38.175297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099449-2202-4d4d-932a-daed14dda6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:38.175617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099449-2202-4d4d-932a-daed14dda6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:38.204467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-a9099449-2202-4d4d-932a-daed14dda6d6 None None] MFA Rules not processed for user `e852eb3e1f574f8a991dc72089a2af36`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:38.205662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a9099449-2202-4d4d-932a-daed14dda6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:38.223190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:38.274064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1284/2559] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:38.285021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:38.286761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:38.343498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567bf29-de24-42a5-a722-caa5d39cd84e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:38.347176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:38.347769 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:38.348053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:38.348328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:38.357643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:38.359114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:38.368649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1276/2560] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:38.597367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9099449-2202-4d4d-932a-daed14dda6d6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:38.607279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9099449-2202-4d4d-932a-daed14dda6d6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:38.611303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ef0f-285f-4d91-8ec9-e9b306383696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:38.611921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ef0f-285f-4d91-8ec9-e9b306383696 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:38.612196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ef0f-285f-4d91-8ec9-e9b306383696 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:38.612493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ef0f-285f-4d91-8ec9-e9b306383696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:38.674446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8ef0f-285f-4d91-8ec9-e9b306383696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:38.675960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8ef0f-285f-4d91-8ec9-e9b306383696 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:38.685353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1285/2561] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:46.935883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:46.935883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:46.985653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:46.998620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ffc3b-0d41-4ae3-ac6f-9b5eb584e011 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:47.003217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:47.003845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:47.004103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:47.004372 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] PATH_INFO: `/v3/users/256b4ce317664b5493e81ab3a5a0f3bf` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:47.005625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=256b4ce317664b5493e81ab3a5a0f3bf)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:47.007658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:47.206385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '256b4ce317664b5493e81ab3a5a0f3bf'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:47.207068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '256b4ce317664b5493e81ab3a5a0f3bf'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:47.207681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '256b4ce317664b5493e81ab3a5a0f3bf'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:47.215281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '256b4ce317664b5493e81ab3a5a0f3bf'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:47.227569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '256b4ce317664b5493e81ab3a5a0f3bf'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:47.313745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1277/2562] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:46 2022] DELETE /identity/v3/users/256b4ce317664b5493e81ab3a5a0f3bf => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:47.318202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:47.318689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:47.318797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:47.337414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:47.338602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:47.395138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1286/2563] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:47.405315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:47.406719 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:47.463207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c62fe-e01e-4602-957f-cdb0a6dfbd74 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:47.466723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:47.467316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:47.467702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:47.467975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:47.476749 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:47.478166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:47.487547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1278/2564] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:47.898813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:47.900214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:47.912609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd53fd-4d97-4053-bac9-69fbc5da68a6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:47.917758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:47.918520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:47.918834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:47.919227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] PATH_INFO: `/v3/projects/bcfccce80ab74e6591df6ef220a26a95` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:47.920409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcfccce80ab74e6591df6ef220a26a95)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:47.922380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:47.981674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:47.983358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:47.996350 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9081bd-c5ff-4502-bee6-7f4c70386156 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.000907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.001546 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.003525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.003525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] PATH_INFO: `/v3/users/ff56d19d3761424dafc57a905880e0f9` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.003525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff56d19d3761424dafc57a905880e0f9)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.005161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.040760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:48.042074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcfccce80ab74e6591df6ef220a26a95', 'request_id': 'req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.047590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcfccce80ab74e6591df6ef220a26a95', 'request_id': 'req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.055142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcfccce80ab74e6591df6ef220a26a95', 'request_id': 'req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.108368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] The token cache is being invalidate because project bcfccce80ab74e6591df6ef220a26a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:07:48.108651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.109263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3b3f83cb-0000-4c8d-8ead-1a1d57fc45d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.111070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1287/2565] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:47 2022] DELETE /identity/v3/projects/bcfccce80ab74e6591df6ef220a26a95 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:48.116998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15935-b8c1-4da6-854c-fb9c476f7a52 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.117279 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15935-b8c1-4da6-854c-fb9c476f7a52 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.117548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15935-b8c1-4da6-854c-fb9c476f7a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.135894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7cc15935-b8c1-4da6-854c-fb9c476f7a52 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:48.137070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7cc15935-b8c1-4da6-854c-fb9c476f7a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:48.155390 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff56d19d3761424dafc57a905880e0f9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.156068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff56d19d3761424dafc57a905880e0f9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.156699 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff56d19d3761424dafc57a905880e0f9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.164450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff56d19d3761424dafc57a905880e0f9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.174978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:48.178392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-7945b8b5-a149-4286-8543-acb41c64d8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff56d19d3761424dafc57a905880e0f9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:48.195761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1288/2566] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:48.203457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46ae67-cf92-47b2-9546-3cc33e21a7cf None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.203802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46ae67-cf92-47b2-9546-3cc33e21a7cf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.204105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46ae67-cf92-47b2-9546-3cc33e21a7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.222968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ee46ae67-cf92-47b2-9546-3cc33e21a7cf None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:48.224071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ee46ae67-cf92-47b2-9546-3cc33e21a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:48.231883 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1289/2567] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:07:48.236613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46ae67-cf92-47b2-9546-3cc33e21a7cf None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.245747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46ae67-cf92-47b2-9546-3cc33e21a7cf None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.250298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.250896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.251172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.251446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.252071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:07:48.252370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.253940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.255484 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1279/2568] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:47 2022] DELETE /identity/v3/users/ff56d19d3761424dafc57a905880e0f9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:07:48.262795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.263531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.263842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.269149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1290/2569] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:07:48.270063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.279634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca706d6-ad2d-46c2-8a6d-7b7fd1433245 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.282778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:48.284003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:48.335544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.336730 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.337008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.337277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.337881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.339786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.342476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1280/2570] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:07:48.352637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.354033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:48.409891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081b9d1-37b3-4fd9-ba29-dafc1a37429c None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.413765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.414493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.414827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.415077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.424960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.426412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.427689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1291/2571] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:48 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:48.432004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.436595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1281/2572] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:07:48.477546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:48.494757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c5e3c-82a8-41ec-a482-f13f14356240 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.499564 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.500168 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.500496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.500772 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.501488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.503591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.592959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1292/2573] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:48 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:48.597010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.606616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785b938-5a2a-411d-a304-409a9a3942fd service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.611449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.612029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.612311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.612581 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.613223 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.614876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.625269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1282/2574] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:48.629361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.639853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0cb6c-9ca5-4bbc-8631-ff76b5331465 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.644566 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87159-39b9-4a4e-93e9-5188fb58123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.645046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87159-39b9-4a4e-93e9-5188fb58123c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.645324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87159-39b9-4a4e-93e9-5188fb58123c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.645595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87159-39b9-4a4e-93e9-5188fb58123c admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.646227 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb87159-39b9-4a4e-93e9-5188fb58123c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.648436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb87159-39b9-4a4e-93e9-5188fb58123c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.658269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1293/2575] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:48.664136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.676505 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ed74a-263d-43fa-97be-6aeaf1887c40 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.681193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.681929 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.682276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.682617 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] PATH_INFO: `/v3/projects/0dca92b240df4087bc2485ae15655370/users/4c1b7bcf74b14a53bbbcb4163058f013/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.699525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dca92b240df4087bc2485ae15655370, user_id=4c1b7bcf74b14a53bbbcb4163058f013, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.702183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.813451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1283/2576] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:48 2022] PUT /identity/v3/projects/0dca92b240df4087bc2485ae15655370/users/4c1b7bcf74b14a53bbbcb4163058f013/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:48.827039 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c64fa-6eb3-4ef3-9906-1f1e15fc37ee None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.827361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c64fa-6eb3-4ef3-9906-1f1e15fc37ee None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.827634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c64fa-6eb3-4ef3-9906-1f1e15fc37ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.855624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-dd2c64fa-6eb3-4ef3-9906-1f1e15fc37ee None None] MFA Rules not processed for user `4c1b7bcf74b14a53bbbcb4163058f013`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:48.856887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-dd2c64fa-6eb3-4ef3-9906-1f1e15fc37ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:48.865362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.868570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:48.936893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1294/2577] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:48.938726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:48.940966 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c64fa-6eb3-4ef3-9906-1f1e15fc37ee None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:48.944035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9112efc-88ad-4a78-a02d-27803d72d0ea admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.950702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.951366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.951649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.951946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] PATH_INFO: `/v3/projects/3cf3d00bd3ff4a00b370d9ee91405884` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.952642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c64fa-6eb3-4ef3-9906-1f1e15fc37ee None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:48.953076 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf3d00bd3ff4a00b370d9ee91405884)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.955018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:48.957473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:48.958090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:48.958369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:48.958642 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:48.959307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:48.961247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.074695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1295/2578] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:48 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:49.078744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.088068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf3d00bd3ff4a00b370d9ee91405884', 'request_id': 'req-c5f17dd0-2967-411c-813e-d751c5ab6720'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.095091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf3d00bd3ff4a00b370d9ee91405884', 'request_id': 'req-c5f17dd0-2967-411c-813e-d751c5ab6720'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.101258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf3d00bd3ff4a00b370d9ee91405884', 'request_id': 'req-c5f17dd0-2967-411c-813e-d751c5ab6720'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.140453 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a906f0-c9f9-4b82-a5d5-d50d8936e28b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:49.144991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:49.145578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.145849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.146116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.146773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:49.148662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.159760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] The token cache is being invalidate because project 3cf3d00bd3ff4a00b370d9ee91405884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:07:49.160057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.160717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.162214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1284/2579] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:48 2022] DELETE /identity/v3/projects/3cf3d00bd3ff4a00b370d9ee91405884 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:49.166627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.168064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:49.229550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f17dd0-2967-411c-813e-d751c5ab6720 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:49.234230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:49.235134 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.235410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.235677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] PATH_INFO: `/v3/users/e852eb3e1f574f8a991dc72089a2af36` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.236813 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e852eb3e1f574f8a991dc72089a2af36)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:49.238743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.247928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1296/2580] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:49 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:49.248932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.253173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:49.262402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e73f34-b4ed-44ec-8d3f-f400a0c554d7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:49.267495 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:49.268080 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.268357 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.268624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.269253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:49.270853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.280660 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1297/2581] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:49.284438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.294412 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a1b85-aef0-4747-be7f-b563ca38bc67 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:49.298895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:49.299425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.299699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.300042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.300635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:49.302113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.303326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:49.313142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1298/2582] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:07:49.318974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.328007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b0479-d5db-48c1-ab36-961f0dadf30e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:49.332459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85000a-acb9-4bd4-813e-9f4b52c66464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:49.333049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85000a-acb9-4bd4-813e-9f4b52c66464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.333324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85000a-acb9-4bd4-813e-9f4b52c66464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.333594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85000a-acb9-4bd4-813e-9f4b52c66464 admin admin] PATH_INFO: `/v3/projects/d56ac85835e440e1a42841ed7ed43d6a/users/3beeb3b5159a4806b0fab99ecf2e5661/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.348388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85000a-acb9-4bd4-813e-9f4b52c66464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56ac85835e440e1a42841ed7ed43d6a, user_id=3beeb3b5159a4806b0fab99ecf2e5661, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:49.352064 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85000a-acb9-4bd4-813e-9f4b52c66464 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.446496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e852eb3e1f574f8a991dc72089a2af36'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.447172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e852eb3e1f574f8a991dc72089a2af36'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.447760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e852eb3e1f574f8a991dc72089a2af36'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.454675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e852eb3e1f574f8a991dc72089a2af36'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.467181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e852eb3e1f574f8a991dc72089a2af36'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:49.468030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1299/2583] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:49 2022] PUT /identity/v3/projects/d56ac85835e440e1a42841ed7ed43d6a/users/3beeb3b5159a4806b0fab99ecf2e5661/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:07:49.483261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899a10-4102-4baf-a40b-533c42ac9d25 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.483544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899a10-4102-4baf-a40b-533c42ac9d25 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.483813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8899a10-4102-4baf-a40b-533c42ac9d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.511387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f8899a10-4102-4baf-a40b-533c42ac9d25 None None] MFA Rules not processed for user `3beeb3b5159a4806b0fab99ecf2e5661`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:49.512582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f8899a10-4102-4baf-a40b-533c42ac9d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:49.564789 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1285/2584] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:49 2022] DELETE /identity/v3/users/e852eb3e1f574f8a991dc72089a2af36 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:07:49.578203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1300/2585] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:07:49.582662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.584126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:49.590742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:49.650598 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df654fdd-183a-43df-90ee-c42c9efc124b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:49.655256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:49.655884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.656163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.656432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.657062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:49.659007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.768049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1286/2586] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:49 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:49.772198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8899a10-4102-4baf-a40b-533c42ac9d25 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.852727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8899a10-4102-4baf-a40b-533c42ac9d25 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:49.924085 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:49.924693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:49.924972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:49.925239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:49.925862 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:49.927814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:49.989442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:49.996029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1301/2587] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:49 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:49.999849 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:49.999991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.001716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:50.012532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5a20-d4c5-4958-b20a-3ccaecf9d743 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.017033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e1443-1112-4117-b5e1-38389c7105ca admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.018458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.019439 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.019722 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.019991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.020625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.022124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.022267 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.022913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.023189 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.023458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] PATH_INFO: `/v3/projects/24682603ccd5412fa9c7a27011bce0a3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.024648 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24682603ccd5412fa9c7a27011bce0a3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.026502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.035170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1287/2588] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:50.039814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.049588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f5f2a-8bf0-429a-a4f7-ed7fcdf8aa74 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.054776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.055414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.055675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.055942 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] PATH_INFO: `/v3/projects/f132f775e82444a5b4bef3cbf6ebcf33/users/235f18a3840a4c34b19714673573ddae/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.070283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f132f775e82444a5b4bef3cbf6ebcf33, user_id=235f18a3840a4c34b19714673573ddae, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.073113 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.131661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1288/2589] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:50 2022] PUT /identity/v3/projects/f132f775e82444a5b4bef3cbf6ebcf33/users/235f18a3840a4c34b19714673573ddae/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:07:50.135559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.138768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24682603ccd5412fa9c7a27011bce0a3', 'request_id': 'req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:50.144233 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24682603ccd5412fa9c7a27011bce0a3', 'request_id': 'req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:50.151024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24682603ccd5412fa9c7a27011bce0a3', 'request_id': 'req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:50.195816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d1dfb-fd2c-4346-9de6-110fad8cccd0 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.212350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] The token cache is being invalidate because project 24682603ccd5412fa9c7a27011bce0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:07:50.212639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:50.217010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-55b7c7b2-6e28-40f6-b15f-d21366f0ace7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:50.217010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1302/2590] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:49 2022] DELETE /identity/v3/projects/24682603ccd5412fa9c7a27011bce0a3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:50.219524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e3ceb-801c-4d15-a3ee-2a77a3c5796a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.219800 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e3ceb-801c-4d15-a3ee-2a77a3c5796a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.220072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e3ceb-801c-4d15-a3ee-2a77a3c5796a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.239827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-198e3ceb-801c-4d15-a3ee-2a77a3c5796a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:50.241026 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-198e3ceb-801c-4d15-a3ee-2a77a3c5796a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:50.254188 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.254821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.255100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.255508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.256103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.257742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.260279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:50.267753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1289/2591] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:50.271794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.274104 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:50.286368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3b57-a501-43a8-a9f2-6e042a1c4ee0 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.292041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a60be-1f7d-4eb4-b87d-f4a0c8e32d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.292748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a60be-1f7d-4eb4-b87d-f4a0c8e32d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.293094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a60be-1f7d-4eb4-b87d-f4a0c8e32d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.293433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a60be-1f7d-4eb4-b87d-f4a0c8e32d5c admin admin] PATH_INFO: `/v3/projects/f132f775e82444a5b4bef3cbf6ebcf33/users/235f18a3840a4c34b19714673573ddae/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.295394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a60be-1f7d-4eb4-b87d-f4a0c8e32d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f132f775e82444a5b4bef3cbf6ebcf33, user_id=235f18a3840a4c34b19714673573ddae, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.298070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a60be-1f7d-4eb4-b87d-f4a0c8e32d5c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.299847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1303/2592] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:07:50.309201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1290/2593] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:50 2022] PUT /identity/v3/projects/f132f775e82444a5b4bef3cbf6ebcf33/users/235f18a3840a4c34b19714673573ddae/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:50.316196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc6cb1-73d5-4fd5-9748-b2220f4dfa29 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.316482 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc6cb1-73d5-4fd5-9748-b2220f4dfa29 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.316782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc6cb1-73d5-4fd5-9748-b2220f4dfa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.318557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.318915 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.319197 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.336960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8cfc6cb1-73d5-4fd5-9748-b2220f4dfa29 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:50.338148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8cfc6cb1-73d5-4fd5-9748-b2220f4dfa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:50.347987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] MFA Rules not processed for user `235f18a3840a4c34b19714673573ddae`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:50.349220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:50.398202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1304/2594] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:50.402262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc6cb1-73d5-4fd5-9748-b2220f4dfa29 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.411074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc6cb1-73d5-4fd5-9748-b2220f4dfa29 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.415882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.417699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.417699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.417699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.417699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:07:50.418023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.419527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.423820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1291/2595] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:07:50.435546 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.436111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1305/2596] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:07:50.442187 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:50.445126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.454469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b518c85-b159-4aaa-8404-741d47a02a47 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.459028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.459659 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.460469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.460743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.461376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.463283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.468576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:50.495308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474104a-604e-4264-89d9-baefcd2e0787 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.499060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.499694 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.499970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.500239 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.509802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.511280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.520599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1292/2597] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:07:50.565341 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1306/2598] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:50 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:50.572045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.582534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87589649-08fb-4fff-be2e-fe8f79a051a8 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.640293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.640899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.641180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.641453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.642090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.644051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.752743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1293/2599] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:50 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:50.757809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.768350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594c9b0-9ece-4dee-b698-f724323409bb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.773421 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.774090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.774369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.774638 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.775304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.776953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.787801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1307/2600] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:50.791872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.797047 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:50.805410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79b31-9c66-4d88-9ff8-93cc9cbb7d5d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.807146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4e72e-da37-47d4-8e5c-6893cbc79df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.807731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4e72e-da37-47d4-8e5c-6893cbc79df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.808007 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4e72e-da37-47d4-8e5c-6893cbc79df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.808392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4e72e-da37-47d4-8e5c-6893cbc79df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.809024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4e72e-da37-47d4-8e5c-6893cbc79df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.810681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4e72e-da37-47d4-8e5c-6893cbc79df9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.821335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1294/2601] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:50.825332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:50.835869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a290e4-ed6a-4aae-981c-255d2d4c6ff8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:50.840556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:50.841190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.841467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.841740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] PATH_INFO: `/v3/projects/c4774c058f224c83bad08e926d4505d7/users/111d80699689422493193dc4f452c784/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.857349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4774c058f224c83bad08e926d4505d7, user_id=111d80699689422493193dc4f452c784, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:50.860297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:50.937643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1308/2602] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:50 2022] PUT /identity/v3/projects/c4774c058f224c83bad08e926d4505d7/users/111d80699689422493193dc4f452c784/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:50.947416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e78d5-7c4c-4775-a04d-6949c153520e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:50.947698 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e78d5-7c4c-4775-a04d-6949c153520e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:50.947973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e78d5-7c4c-4775-a04d-6949c153520e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:50.976315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a62e78d5-7c4c-4775-a04d-6949c153520e None None] MFA Rules not processed for user `111d80699689422493193dc4f452c784`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:50.977691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a62e78d5-7c4c-4775-a04d-6949c153520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:51.041005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1295/2603] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:51.045635 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.104748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9cf7c-3282-4bec-911e-66c097e08091 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.109352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:51.110204 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.110476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.110777 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.111388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:51.113304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:51.243557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1309/2604] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:51 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:51.247527 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e78d5-7c4c-4775-a04d-6949c153520e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.259159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e78d5-7c4c-4775-a04d-6949c153520e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.313580 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:51.314099 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.314375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.314640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.315282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:51.317171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:51.411042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1296/2605] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:51 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:07:51.415575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.426734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ce7d-e4fc-49e7-b451-71de144d6396 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.432438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:51.433109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.433390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.433663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.434301 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:51.435918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:51.446993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1310/2606] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:51.450768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.461597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5060e7e-bc8c-45b6-9187-e7a0845d99b7 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.466395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:51.467019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.467301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.467590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.468212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:51.469807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:51.480324 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1297/2607] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:07:51.489359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.498572 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9158390-2cb0-45b1-899b-1e9ab8cd4cf1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.503229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3973b3c-7bd2-4571-b39b-642ae6c1601d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:51.503873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3973b3c-7bd2-4571-b39b-642ae6c1601d admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.504151 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3973b3c-7bd2-4571-b39b-642ae6c1601d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.504422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3973b3c-7bd2-4571-b39b-642ae6c1601d admin admin] PATH_INFO: `/v3/projects/5c553fdbb3f54ed4b161feeb1ee2cf93/users/751497a4de8c40659d0f575dfc7b071e/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.519470 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3973b3c-7bd2-4571-b39b-642ae6c1601d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c553fdbb3f54ed4b161feeb1ee2cf93, user_id=751497a4de8c40659d0f575dfc7b071e, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:51.522069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3973b3c-7bd2-4571-b39b-642ae6c1601d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:51.593805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.601406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1311/2608] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:51 2022] PUT /identity/v3/projects/5c553fdbb3f54ed4b161feeb1ee2cf93/users/751497a4de8c40659d0f575dfc7b071e/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:51.612913 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2f028-154c-446e-9f5b-c11ba6fe0d30 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.613213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2f028-154c-446e-9f5b-c11ba6fe0d30 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.613538 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2f028-154c-446e-9f5b-c11ba6fe0d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.641099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2ca2f028-154c-446e-9f5b-c11ba6fe0d30 None None] MFA Rules not processed for user `751497a4de8c40659d0f575dfc7b071e`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:51.642314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2ca2f028-154c-446e-9f5b-c11ba6fe0d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:51.653352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0023a4-e8fb-4999-a9d0-c018281d968a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.666275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:51.694197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:51.703050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:51.703655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.703933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.704204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.705960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:51.710874 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1312/2609] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:07:51.715486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2f028-154c-446e-9f5b-c11ba6fe0d30 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.765143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:51.766715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:51.776170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1298/2610] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:07:51.779155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2f028-154c-446e-9f5b-c11ba6fe0d30 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.783752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:51.784345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:51.784620 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:51.784912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:51.785584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:51.787499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:51.914167 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1313/2611] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:07:51 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:51.918249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:51.929206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dd8ab-e0bc-4c64-aed0-c00a8d4c306f service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:51.999602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:52.000212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.000492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.000767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.001386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:52.003374 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:52.098203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1299/2612] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:07:51 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:07:52.102193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:52.112812 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c912d0b-6969-46da-b2c9-eca8f2a951c3 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:52.118249 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:52.118871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.125372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.125372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.125372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:52.125372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:52.131442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1314/2613] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:07:52.135481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:52.145277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400171a9-4c1d-418c-8f09-b39573d64ede admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:52.151077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:52.151761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.154160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.154160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] PATH_INFO: `/v3/projects/e8df579bad374c6a857e34f513e2fc84/users/aa5d705424df4599b8be061d5da618dd/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.167963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8df579bad374c6a857e34f513e2fc84, user_id=aa5d705424df4599b8be061d5da618dd, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:52.170629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:52.267548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1300/2614] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:52 2022] PUT /identity/v3/projects/e8df579bad374c6a857e34f513e2fc84/users/aa5d705424df4599b8be061d5da618dd/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:07:52.271569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:52.281377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9c4bf-40a1-4a1f-b4ad-9c2a75a3bcff admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:52.341684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a68568-7c08-4a39-8ede-77435ab2b0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:52.341684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a68568-7c08-4a39-8ede-77435ab2b0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.341684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a68568-7c08-4a39-8ede-77435ab2b0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.341684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a68568-7c08-4a39-8ede-77435ab2b0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.342017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a68568-7c08-4a39-8ede-77435ab2b0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:52.342017 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a68568-7c08-4a39-8ede-77435ab2b0e6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:52.351855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1315/2615] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:07:52.356176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:52.369340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f87c0-2e89-49d2-9dba-db866d2b22f0 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:52.374388 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:52.375370 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.375650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.376288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] PATH_INFO: `/v3/projects/e8df579bad374c6a857e34f513e2fc84/users/aa5d705424df4599b8be061d5da618dd/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.378276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8df579bad374c6a857e34f513e2fc84, user_id=aa5d705424df4599b8be061d5da618dd, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:52.380954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:52.396251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1301/2616] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:07:52 2022] PUT /identity/v3/projects/e8df579bad374c6a857e34f513e2fc84/users/aa5d705424df4599b8be061d5da618dd/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:52.406403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.406747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.407051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.439509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] MFA Rules not processed for user `aa5d705424df4599b8be061d5da618dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:52.440744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:52.489967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:52.510434 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1316/2617] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:07:52.529045 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:52.556521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:52.586766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4871692-fc24-448b-8254-20b64fca0a93 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:52.590628 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:52.591261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.591539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.591811 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.601946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:52.603410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:52.613520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1302/2618] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:52.705925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:52.707401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:52.719783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824374e6-45df-4a23-a5b1-a414ff12388f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:52.723337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:52.723928 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:52.724203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:52.724475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:52.780271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:52.781735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:52.790525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1317/2619] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:56.682165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:56.690902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a503d3-efa8-4299-804d-55685af9dbc9 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:56.694983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907868c-afa2-469e-a066-32b769bb0587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:56.695631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907868c-afa2-469e-a066-32b769bb0587 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:56.696132 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907868c-afa2-469e-a066-32b769bb0587 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:56.696301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907868c-afa2-469e-a066-32b769bb0587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:56.697965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9907868c-afa2-469e-a066-32b769bb0587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:56.763712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907868c-afa2-469e-a066-32b769bb0587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:56.765145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907868c-afa2-469e-a066-32b769bb0587 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:56.774709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1303/2620] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:56.866545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:56.930272 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704504f0-dc6c-4c7c-9e50-df21e4fc6b15 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:56.934887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:56.935505 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:56.935782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:56.936051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] PATH_INFO: `/v3/users/111d80699689422493193dc4f452c784` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:56.937170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111d80699689422493193dc4f452c784)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:56.939107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:57.144171 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111d80699689422493193dc4f452c784'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:57.144783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111d80699689422493193dc4f452c784'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:57.246413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111d80699689422493193dc4f452c784'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:57.260536 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111d80699689422493193dc4f452c784'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:57.261175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111d80699689422493193dc4f452c784'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:57.273015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1318/2621] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:56 2022] DELETE /identity/v3/users/111d80699689422493193dc4f452c784 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:57.278319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:57.278608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:57.278913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:57.297200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:07:57.298400 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:57.357966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1304/2622] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:07:57.373840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:57.375300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:57.381705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:57.433971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6dd7-d433-4a05-b60e-688cf98a9a99 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:57.437631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:57.438222 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:57.438493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:57.438817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:57.448308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:57.449725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:57.459037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1319/2623] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:07:57.853754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:57.855234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:57.874602 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2b6cb-1621-4d39-9e65-2389047c9184 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:57.880809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:57.881031 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:57.881182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:57.881544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] PATH_INFO: `/v3/projects/c4774c058f224c83bad08e926d4505d7` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:57.882902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4774c058f224c83bad08e926d4505d7)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:57.885032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:57.993450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4774c058f224c83bad08e926d4505d7', 'request_id': 'req-57b312f4-cebe-4103-9ba3-0543ae2e8061'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:57.999579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4774c058f224c83bad08e926d4505d7', 'request_id': 'req-57b312f4-cebe-4103-9ba3-0543ae2e8061'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.005037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:58.007201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4774c058f224c83bad08e926d4505d7', 'request_id': 'req-57b312f4-cebe-4103-9ba3-0543ae2e8061'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.086264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] The token cache is being invalidate because project c4774c058f224c83bad08e926d4505d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:07:58.086264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.086540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.088013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1305/2624] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:57 2022] DELETE /identity/v3/projects/c4774c058f224c83bad08e926d4505d7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:58.092136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:58.102605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729de4ac-0e7d-415d-b6e9-7e6ae3c0f8e8 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:58.163854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:58.164520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:58.164774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:58.165046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] PATH_INFO: `/v3/users/751497a4de8c40659d0f575dfc7b071e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:58.166152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=751497a4de8c40659d0f575dfc7b071e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:58.168110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:58.340605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751497a4de8c40659d0f575dfc7b071e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.341246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751497a4de8c40659d0f575dfc7b071e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.429213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751497a4de8c40659d0f575dfc7b071e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.441223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751497a4de8c40659d0f575dfc7b071e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.441862 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751497a4de8c40659d0f575dfc7b071e'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:58.454432 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1320/2625] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:58 2022] DELETE /identity/v3/users/751497a4de8c40659d0f575dfc7b071e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:58.914703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:58.916161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:58.929755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b312f4-cebe-4103-9ba3-0543ae2e8061 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:58.985750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:58.986379 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:58.986675 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:58.986955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] PATH_INFO: `/v3/projects/5c553fdbb3f54ed4b161feeb1ee2cf93` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:58.988226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c553fdbb3f54ed4b161feeb1ee2cf93)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:58.990160 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:59.137606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c553fdbb3f54ed4b161feeb1ee2cf93', 'request_id': 'req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.143585 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c553fdbb3f54ed4b161feeb1ee2cf93', 'request_id': 'req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.149039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c553fdbb3f54ed4b161feeb1ee2cf93', 'request_id': 'req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.222234 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] The token cache is being invalidate because project 5c553fdbb3f54ed4b161feeb1ee2cf93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:07:59.222520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.223159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.224674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1306/2626] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:58 2022] DELETE /identity/v3/projects/5c553fdbb3f54ed4b161feeb1ee2cf93 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:07:59.229232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:59.230682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:59.243496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecb4c7-6ff3-40d8-88f2-592fb4ef9fd1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:07:59.267915 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:07:59.299930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:07:59.300574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:07:59.300855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:07:59.301124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] PATH_INFO: `/v3/users/aa5d705424df4599b8be061d5da618dd` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:07:59.302259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5d705424df4599b8be061d5da618dd)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:07:59.304269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:07:59.472093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5d705424df4599b8be061d5da618dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.472754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5d705424df4599b8be061d5da618dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.531646 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5d705424df4599b8be061d5da618dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.544542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5d705424df4599b8be061d5da618dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.545181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-94ad893f-3bbe-4c11-9bdd-e290a5b3e5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5d705424df4599b8be061d5da618dd'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:07:59.557477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1321/2627] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:07:59 2022] DELETE /identity/v3/users/aa5d705424df4599b8be061d5da618dd => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:07:59.976968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:07:59.978407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:07:59.995035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f473e-e77b-4e69-9c56-ccc3142a3ddc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.051494 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.052130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.052392 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.052658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] PATH_INFO: `/v3/projects/e8df579bad374c6a857e34f513e2fc84` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.054261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8df579bad374c6a857e34f513e2fc84)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.056257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.077298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:00.155140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8df579bad374c6a857e34f513e2fc84', 'request_id': 'req-98638a1d-b682-451b-b888-3c1b748a4b3f'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:00.161198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8df579bad374c6a857e34f513e2fc84', 'request_id': 'req-98638a1d-b682-451b-b888-3c1b748a4b3f'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:00.166787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8df579bad374c6a857e34f513e2fc84', 'request_id': 'req-98638a1d-b682-451b-b888-3c1b748a4b3f'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:00.228752 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] The token cache is being invalidate because project e8df579bad374c6a857e34f513e2fc84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:00.228997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:00.229657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-98638a1d-b682-451b-b888-3c1b748a4b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:00.231222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1307/2628] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:07:59 2022] DELETE /identity/v3/projects/e8df579bad374c6a857e34f513e2fc84 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:00.238672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bbab5-7a94-4096-9fda-5723bfb9226f None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.239172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bbab5-7a94-4096-9fda-5723bfb9226f None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.239258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bbab5-7a94-4096-9fda-5723bfb9226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.259338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-264bbab5-7a94-4096-9fda-5723bfb9226f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:00.260545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-264bbab5-7a94-4096-9fda-5723bfb9226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:00.319943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1322/2629] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:00.328610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f491d4-a1b4-4e3f-8907-bc99adcabcc6 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.328889 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f491d4-a1b4-4e3f-8907-bc99adcabcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.329183 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f491d4-a1b4-4e3f-8907-bc99adcabcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.349090 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-05f491d4-a1b4-4e3f-8907-bc99adcabcc6 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:00.350283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-05f491d4-a1b4-4e3f-8907-bc99adcabcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:00.358894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1308/2630] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:00.365615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bbab5-7a94-4096-9fda-5723bfb9226f None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:00.375357 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bbab5-7a94-4096-9fda-5723bfb9226f None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.379870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.380680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.381040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.381299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.382414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:08:00.382745 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.384327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.395826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1323/2631] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:08:00.400846 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f491d4-a1b4-4e3f-8907-bc99adcabcc6 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:00.410144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f491d4-a1b4-4e3f-8907-bc99adcabcc6 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.414922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.415512 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.415794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.416065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.416684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.418613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.532136 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1309/2632] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:08:00 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:08:00.543688 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:00.553666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32978855-f544-4ec4-a4f6-896608b8478d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.583235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:00.609441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.610295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.610573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.610872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.611493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.613400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.726035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1324/2633] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:08:00 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:00.730320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:00.740961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb0fa1-0a85-437a-831d-c21976dd7077 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.746225 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.746790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.747096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.747371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.748006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.749595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.763286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1310/2634] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:00.765631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:00.776521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6506f-2365-4429-820e-bb849e9b0328 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.781139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.781729 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.782001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.782271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.782929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.784555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.794890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1325/2635] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:00.798916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:00.809520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3c762-cff7-4329-9089-d8dd94e43ed4 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.816152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:00.817054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9bff6-3e9a-40bf-980a-645260ed54a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.817751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9bff6-3e9a-40bf-980a-645260ed54a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.818062 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9bff6-3e9a-40bf-980a-645260ed54a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.818335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9bff6-3e9a-40bf-980a-645260ed54a0 admin admin] PATH_INFO: `/v3/projects/262256e7068b41fa96dde68614cfc126/users/7eb2f527351f49f4aef0be7535f60baa/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.820562 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:00.833236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cac449-16e9-4556-bdee-009324e8a1f6 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:00.834157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9bff6-3e9a-40bf-980a-645260ed54a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=262256e7068b41fa96dde68614cfc126, user_id=7eb2f527351f49f4aef0be7535f60baa, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.836886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9bff6-3e9a-40bf-980a-645260ed54a0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.837795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:00.838430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.838733 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.839004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] PATH_INFO: `/v3/users/4c1b7bcf74b14a53bbbcb4163058f013` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.840096 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1b7bcf74b14a53bbbcb4163058f013)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:00.841965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:00.908625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1311/2636] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:00 2022] PUT /identity/v3/projects/262256e7068b41fa96dde68614cfc126/users/7eb2f527351f49f4aef0be7535f60baa/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:00.913573 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:00.913861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:00.914137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:00.932423 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:00.933611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:00.991809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1312/2637] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:01.003082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:01.004416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:01.027988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1b7bcf74b14a53bbbcb4163058f013'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:01.028825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1b7bcf74b14a53bbbcb4163058f013'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:01.053046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:01.064347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287f4f-0215-4d15-9b4d-184972e5836f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:01.067958 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb143a09-0ef5-4c76-bbe4-4526dd4f9492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:01.068787 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb143a09-0ef5-4c76-bbe4-4526dd4f9492 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:01.069166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb143a09-0ef5-4c76-bbe4-4526dd4f9492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:01.070033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb143a09-0ef5-4c76-bbe4-4526dd4f9492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:01.078944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb143a09-0ef5-4c76-bbe4-4526dd4f9492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:01.080358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb143a09-0ef5-4c76-bbe4-4526dd4f9492 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:01.089535 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1313/2638] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:08:01.120433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1b7bcf74b14a53bbbcb4163058f013'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:01.132767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1b7bcf74b14a53bbbcb4163058f013'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:01.133418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1b7bcf74b14a53bbbcb4163058f013'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:01.145770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1326/2639] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:00 2022] DELETE /identity/v3/users/4c1b7bcf74b14a53bbbcb4163058f013 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:08:01.150158 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:01.150441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:01.150732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:01.169415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:01.173469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:01.227618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1314/2640] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:01.237781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:01.239250 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:01.306677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39faae94-6382-4a39-bac9-589c092fd289 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:01.310430 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:01.311060 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:01.311334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:01.311605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:01.325977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:01.327446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:01.337130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1327/2641] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:08:01.829108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:01.830529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:01.843879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1a89-f15e-43c2-a98e-f39d6bfdbd16 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:01.848804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:01.849498 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:01.851243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:01.851243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] PATH_INFO: `/v3/projects/0dca92b240df4087bc2485ae15655370` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:01.852387 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dca92b240df4087bc2485ae15655370)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:01.854302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:01.950893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dca92b240df4087bc2485ae15655370', 'request_id': 'req-ce222494-4673-4b1e-a090-56cb91fbc4a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:01.957595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dca92b240df4087bc2485ae15655370', 'request_id': 'req-ce222494-4673-4b1e-a090-56cb91fbc4a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:01.964207 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dca92b240df4087bc2485ae15655370', 'request_id': 'req-ce222494-4673-4b1e-a090-56cb91fbc4a3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.011684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] The token cache is being invalidate because project 0dca92b240df4087bc2485ae15655370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:02.012291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.012836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ce222494-4673-4b1e-a090-56cb91fbc4a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.014375 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1315/2642] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:01 2022] DELETE /identity/v3/projects/0dca92b240df4087bc2485ae15655370 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:02.018477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:02.029517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d6b6f-f046-4b38-a950-2f9dd660ffe1 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:02.057349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:02.092872 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:02.093514 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:02.093786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:02.094052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] PATH_INFO: `/v3/users/3beeb3b5159a4806b0fab99ecf2e5661` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:02.095334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3beeb3b5159a4806b0fab99ecf2e5661)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:02.097256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:02.293354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3beeb3b5159a4806b0fab99ecf2e5661'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.294045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3beeb3b5159a4806b0fab99ecf2e5661'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.350361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:02.354118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:02.354118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:02.370040 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3beeb3b5159a4806b0fab99ecf2e5661'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.382493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] MFA Rules not processed for user `7eb2f527351f49f4aef0be7535f60baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:02.383823 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:02.385103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3beeb3b5159a4806b0fab99ecf2e5661'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.385826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3beeb3b5159a4806b0fab99ecf2e5661'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:02.404741 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1328/2643] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:02 2022] DELETE /identity/v3/users/3beeb3b5159a4806b0fab99ecf2e5661 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:02.466599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:02.470985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1316/2644] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:02.481619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:02.483044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:02.541904 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efb688-66de-403c-848a-807be97db3dc admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:02.545476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:02.546067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:02.546340 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:02.546606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:02.605034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:02.606603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:02.619575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1329/2645] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:02.713610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:02.830504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:02.831945 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:02.854049 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b70bc-0479-4645-8ec5-50ee32ed4823 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:02.909274 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:02.909901 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:02.910168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:02.910437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] PATH_INFO: `/v3/projects/d56ac85835e440e1a42841ed7ed43d6a` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:02.911588 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56ac85835e440e1a42841ed7ed43d6a)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:02.913504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:03.038926 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56ac85835e440e1a42841ed7ed43d6a', 'request_id': 'req-8f9ca07e-3d37-446e-bc77-527ab535d4fd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.045200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56ac85835e440e1a42841ed7ed43d6a', 'request_id': 'req-8f9ca07e-3d37-446e-bc77-527ab535d4fd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.051046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56ac85835e440e1a42841ed7ed43d6a', 'request_id': 'req-8f9ca07e-3d37-446e-bc77-527ab535d4fd'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.158676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] The token cache is being invalidate because project d56ac85835e440e1a42841ed7ed43d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:03.158981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.160137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.161150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1317/2646] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:02 2022] DELETE /identity/v3/projects/d56ac85835e440e1a42841ed7ed43d6a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:03.165229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:03.175644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3c37-30f5-4976-a58e-7fdea6a13bcf service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:03.230658 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:03.231300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:03.231583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:03.231877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] PATH_INFO: `/v3/users/235f18a3840a4c34b19714673573ddae` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:03.232985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235f18a3840a4c34b19714673573ddae)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:03.234922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:03.421183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235f18a3840a4c34b19714673573ddae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.421945 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235f18a3840a4c34b19714673573ddae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.535840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235f18a3840a4c34b19714673573ddae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.548936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235f18a3840a4c34b19714673573ddae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.549595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-afd7e83c-cf55-4dd3-9349-9b44796b4b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235f18a3840a4c34b19714673573ddae'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:03.561954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1330/2647] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:03 2022] DELETE /identity/v3/users/235f18a3840a4c34b19714673573ddae => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:03.927230 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:03.928805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:03.941886 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ca07e-3d37-446e-bc77-527ab535d4fd admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:03.998899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:03.999522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:03.999800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.000073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] PATH_INFO: `/v3/projects/f132f775e82444a5b4bef3cbf6ebcf33` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.001172 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f132f775e82444a5b4bef3cbf6ebcf33)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:04.003178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:04.120414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:04.132321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f132f775e82444a5b4bef3cbf6ebcf33', 'request_id': 'req-2eb038fd-4df2-469e-94d0-1d75f514368d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:04.138973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f132f775e82444a5b4bef3cbf6ebcf33', 'request_id': 'req-2eb038fd-4df2-469e-94d0-1d75f514368d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:04.144397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f132f775e82444a5b4bef3cbf6ebcf33', 'request_id': 'req-2eb038fd-4df2-469e-94d0-1d75f514368d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:04.200003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] The token cache is being invalidate because project f132f775e82444a5b4bef3cbf6ebcf33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:04.200224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:04.200868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2eb038fd-4df2-469e-94d0-1d75f514368d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:04.202376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1318/2648] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:03 2022] DELETE /identity/v3/projects/f132f775e82444a5b4bef3cbf6ebcf33 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:04.208276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07672d-3b07-4da2-a24d-637a73737e9a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.217700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07672d-3b07-4da2-a24d-637a73737e9a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.217700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07672d-3b07-4da2-a24d-637a73737e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.228517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9e07672d-3b07-4da2-a24d-637a73737e9a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:04.229750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9e07672d-3b07-4da2-a24d-637a73737e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:04.287319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1331/2649] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:04.295481 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7365d0-1478-479d-979d-909eb0e0e958 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.295764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7365d0-1478-479d-979d-909eb0e0e958 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.296070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7365d0-1478-479d-979d-909eb0e0e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.315236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-3c7365d0-1478-479d-979d-909eb0e0e958 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:04.316428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3c7365d0-1478-479d-979d-909eb0e0e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:04.324844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1319/2650] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:04.328907 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07672d-3b07-4da2-a24d-637a73737e9a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:04.340607 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07672d-3b07-4da2-a24d-637a73737e9a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:04.345386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:04.349541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.349541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.349541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.349541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:08:04.349541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:04.349541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:04.361667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1332/2651] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:08:04.366522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7365d0-1478-479d-979d-909eb0e0e958 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:04.376278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7365d0-1478-479d-979d-909eb0e0e958 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:04.384150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:04.384150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.384150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.384150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.384488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:04.385414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:04.504953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1320/2652] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:08:04 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:04.509834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:04.519848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f108b27-247e-424c-b1e1-9ad7ee69408a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:04.523377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:04.575266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:04.575850 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.576126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.576440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.577042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:04.578943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:04.680388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1333/2653] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:08:04 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:04.684618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:04.695517 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327fe16-cc02-469c-b873-46055754ac48 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:04.700353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:04.700939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.701216 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.701515 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.702150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:04.703819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:04.713584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1321/2654] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:04.717283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:04.727213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6d77-4a48-4cb5-9705-f1cad5da8def admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:04.732092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6144b01-8f7d-4666-9ccc-764724f2d1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:04.732603 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144b01-8f7d-4666-9ccc-764724f2d1ea admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.732877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144b01-8f7d-4666-9ccc-764724f2d1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.733143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144b01-8f7d-4666-9ccc-764724f2d1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.733773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6144b01-8f7d-4666-9ccc-764724f2d1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:04.735700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6144b01-8f7d-4666-9ccc-764724f2d1ea admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:04.750248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1334/2655] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:04.751645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:04.762570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3f3f-2a16-4dd5-85d4-dd16a12574df admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:04.767258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:04.767920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.768201 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.768596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] PATH_INFO: `/v3/projects/1f03126d63cb4a249a749ccae300cc33/users/9e553a7086174bf484fc89d07fcce336/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.783709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f03126d63cb4a249a749ccae300cc33, user_id=9e553a7086174bf484fc89d07fcce336, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:04.786309 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:04.874103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1322/2656] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:04 2022] PUT /identity/v3/projects/1f03126d63cb4a249a749ccae300cc33/users/9e553a7086174bf484fc89d07fcce336/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:04.886768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8f53-2f22-4c1f-87e6-a1db5204a802 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:04.887048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8f53-2f22-4c1f-87e6-a1db5204a802 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:04.887317 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8f53-2f22-4c1f-87e6-a1db5204a802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:04.915359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c5ce8f53-2f22-4c1f-87e6-a1db5204a802 None None] MFA Rules not processed for user `9e553a7086174bf484fc89d07fcce336`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:04.916560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c5ce8f53-2f22-4c1f-87e6-a1db5204a802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:04.981881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1335/2657] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:04.994097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:04.995861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:05.050906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae06d-9595-4c3c-a36b-8f625e674168 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:05.054479 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8722f-7a93-4aa6-aa8e-241a589ed457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:05.055102 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8722f-7a93-4aa6-aa8e-241a589ed457 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:05.055377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8722f-7a93-4aa6-aa8e-241a589ed457 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:05.055644 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8722f-7a93-4aa6-aa8e-241a589ed457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:05.063674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8722f-7a93-4aa6-aa8e-241a589ed457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:05.065079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8722f-7a93-4aa6-aa8e-241a589ed457 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:05.074096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1323/2658] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:05.093274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8f53-2f22-4c1f-87e6-a1db5204a802 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:05.154166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8f53-2f22-4c1f-87e6-a1db5204a802 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:05.158705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:05.159309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:05.159582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:05.159848 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] PATH_INFO: `/v3/users/9e553a7086174bf484fc89d07fcce336` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:05.160940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e553a7086174bf484fc89d07fcce336)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:05.162864 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:05.177713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:05.366895 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e553a7086174bf484fc89d07fcce336'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:05.367485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e553a7086174bf484fc89d07fcce336'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:05.446674 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e553a7086174bf484fc89d07fcce336'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:05.450934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:05.451415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:05.451506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:05.459472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e553a7086174bf484fc89d07fcce336'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:05.460940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-73a93013-100e-4236-bbe7-e74554ed9a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e553a7086174bf484fc89d07fcce336'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:05.470768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] MFA Rules not processed for user `7eb2f527351f49f4aef0be7535f60baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:05.471968 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:05.472890 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1336/2659] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:05 2022] DELETE /identity/v3/users/9e553a7086174bf484fc89d07fcce336 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:08:05.477251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37abb1-8a17-4e6f-bec7-a207564f1b58 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:05.477530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37abb1-8a17-4e6f-bec7-a207564f1b58 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:05.477794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37abb1-8a17-4e6f-bec7-a207564f1b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:05.497014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9a37abb1-8a17-4e6f-bec7-a207564f1b58 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:05.498205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9a37abb1-8a17-4e6f-bec7-a207564f1b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:05.528931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1324/2660] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:05.557718 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1337/2661] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:05.568024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:05.570077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:05.576427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:05.629695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a9a3f-a651-446f-87a9-ec4f717d85cd None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:05.634561 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:05.635240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:05.635519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:05.635790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:05.644981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:05.646993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:05.657685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1325/2662] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:05.830182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37abb1-8a17-4e6f-bec7-a207564f1b58 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:05.844289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37abb1-8a17-4e6f-bec7-a207564f1b58 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:05.848016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:05.848636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:05.848944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:05.849241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:05.851097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:05.861091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:05.862567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:05.870479 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1338/2663] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:06.160648 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.162357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:06.191249 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7e0eb-4c9d-4b58-82df-d4d58feb4fa5 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:06.195937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:06.196562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.196841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.197114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] PATH_INFO: `/v3/projects/1f03126d63cb4a249a749ccae300cc33` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.198224 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f03126d63cb4a249a749ccae300cc33)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:06.200164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:06.296425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.297858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:06.318767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f03126d63cb4a249a749ccae300cc33', 'request_id': 'req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:06.325661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f03126d63cb4a249a749ccae300cc33', 'request_id': 'req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:06.333039 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f03126d63cb4a249a749ccae300cc33', 'request_id': 'req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:06.372314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd284a-8f04-4a1b-8881-4cc065aa8a52 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:06.392770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:06.423077 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] The token cache is being invalidate because project 1f03126d63cb4a249a749ccae300cc33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:06.423292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:06.423997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-c429b9a4-8c7e-4d48-b0ea-9be6d5b478ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:06.430002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1326/2664] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:06 2022] DELETE /identity/v3/projects/1f03126d63cb4a249a749ccae300cc33 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:06.431544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeebd4-5d49-43dd-858b-7f61d1d73aa5 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.431834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeebd4-5d49-43dd-858b-7f61d1d73aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.432114 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeebd4-5d49-43dd-858b-7f61d1d73aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.436309 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a23ef-8784-425f-9da6-3236d49b5a15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:06.436918 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a23ef-8784-425f-9da6-3236d49b5a15 service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.437201 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a23ef-8784-425f-9da6-3236d49b5a15 service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.437469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a23ef-8784-425f-9da6-3236d49b5a15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.439256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3f9a23ef-8784-425f-9da6-3236d49b5a15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:06.451797 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a23ef-8784-425f-9da6-3236d49b5a15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:06.452185 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-bcfeebd4-5d49-43dd-858b-7f61d1d73aa5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:06.453388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a23ef-8784-425f-9da6-3236d49b5a15 service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:06.453492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-bcfeebd4-5d49-43dd-858b-7f61d1d73aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:06.464182 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1339/2665] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:06.513096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1327/2666] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:06.520871 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f937-2907-4faf-b240-a7634bf21220 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.521155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f937-2907-4faf-b240-a7634bf21220 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.521429 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f937-2907-4faf-b240-a7634bf21220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.541972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1015f937-2907-4faf-b240-a7634bf21220 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:06.543334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1015f937-2907-4faf-b240-a7634bf21220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:06.551457 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1340/2667] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:06.555770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfeebd4-5d49-43dd-858b-7f61d1d73aa5 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.565566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfeebd4-5d49-43dd-858b-7f61d1d73aa5 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:06.570879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:06.571471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.571748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.572018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.572671 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:08:06.572970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:06.574557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:06.586950 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1328/2668] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:08:06.591444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015f937-2907-4faf-b240-a7634bf21220 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.601556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015f937-2907-4faf-b240-a7634bf21220 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:06.606611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:06.607244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.607549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.607821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.608435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:06.610627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:06.726239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1341/2669] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:08:06 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:06.730453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.740700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528180de-1f1f-4bcc-a334-4c424866e079 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:06.753179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:06.796818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:06.797411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.797686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.797957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.798615 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:06.800501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:06.909571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1329/2670] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:08:06 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:06.916709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.927019 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e7d1b-902d-4ecd-b5b6-085ea7df6752 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:06.931695 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:06.932879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.932879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.932879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.933499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:06.935176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:06.945663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1342/2671] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:06.949775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.960376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2068b25-7eca-4193-ad74-f5e882d134d6 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:06.965961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e1857-be3f-4d90-bcd2-c6a313b19321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:06.966503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1857-be3f-4d90-bcd2-c6a313b19321 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:06.966818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1857-be3f-4d90-bcd2-c6a313b19321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:06.967096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e1857-be3f-4d90-bcd2-c6a313b19321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:06.967733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e1857-be3f-4d90-bcd2-c6a313b19321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:06.969342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e1857-be3f-4d90-bcd2-c6a313b19321 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:06.982325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1330/2672] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:06.985570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:06.995707 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d0200-70d2-4593-beb9-c9dd3568c472 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:07.000363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:07.001879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:07.001879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:07.001879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] PATH_INFO: `/v3/projects/22534fb887a943319c1ee0b1e9512de7/users/42d0bd0ef1ee46fa9169513131d6a83f/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:07.016135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=22534fb887a943319c1ee0b1e9512de7, user_id=42d0bd0ef1ee46fa9169513131d6a83f, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:07.018809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:07.125199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1343/2673] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:06 2022] PUT /identity/v3/projects/22534fb887a943319c1ee0b1e9512de7/users/42d0bd0ef1ee46fa9169513131d6a83f/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:07.136232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:07.136518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:07.136794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:07.164518 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] MFA Rules not processed for user `42d0bd0ef1ee46fa9169513131d6a83f`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:07.165753 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:07.230351 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1331/2674] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:07.242597 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:07.244359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:07.300653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d4c9-7d37-4943-aefe-c933c2a7c6fc admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:07.305023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ba4ad-88a8-4a96-8f1d-789c12cd2481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:07.305624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ba4ad-88a8-4a96-8f1d-789c12cd2481 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:07.305905 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ba4ad-88a8-4a96-8f1d-789c12cd2481 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:07.306181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ba4ad-88a8-4a96-8f1d-789c12cd2481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:07.314789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ba4ad-88a8-4a96-8f1d-789c12cd2481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:07.316226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ba4ad-88a8-4a96-8f1d-789c12cd2481 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:07.325100 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1344/2675] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:17.227591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:17.320978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=74,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:24.040511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:24.041985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:24.130532 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41b75f-20ec-406a-82d2-6bb5566e9b1d None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:24.137283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:24.137894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:24.138256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:24.140018 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:24.140342 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:24.153896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:24.155608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:24.167363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1332/2676] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:34.162308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=41 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:35.800329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da92548-ade0-4a41-840f-71e953a3fd09 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:35.800835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da92548-ade0-4a41-840f-71e953a3fd09 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:35.800925 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da92548-ade0-4a41-840f-71e953a3fd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:35.819932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-8da92548-ade0-4a41-840f-71e953a3fd09 None None] MFA Rules not processed for user `7eb2f527351f49f4aef0be7535f60baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:35.821203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8da92548-ade0-4a41-840f-71e953a3fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:35.879917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1345/2677] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:36.109768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:36.111624 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:36.168459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cc131-8357-440c-a3bd-dfd0e6d59817 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:36.172145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9564c97-1556-4fcd-9e97-19dd4398beb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:36.172741 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9564c97-1556-4fcd-9e97-19dd4398beb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:36.173084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9564c97-1556-4fcd-9e97-19dd4398beb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:36.173344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9564c97-1556-4fcd-9e97-19dd4398beb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:36.175137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d9564c97-1556-4fcd-9e97-19dd4398beb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:36.185476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9564c97-1556-4fcd-9e97-19dd4398beb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:36.187006 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9564c97-1556-4fcd-9e97-19dd4398beb7 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:36.195117 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1333/2678] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:38.688815 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da92548-ade0-4a41-840f-71e953a3fd09 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:38.752256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da92548-ade0-4a41-840f-71e953a3fd09 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:38.756876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:38.757493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:38.759535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:38.759535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] PATH_INFO: `/v3/users/42d0bd0ef1ee46fa9169513131d6a83f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:38.759535 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d0bd0ef1ee46fa9169513131d6a83f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:38.762044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:38.969803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d0bd0ef1ee46fa9169513131d6a83f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:38.970464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d0bd0ef1ee46fa9169513131d6a83f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:39.086316 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d0bd0ef1ee46fa9169513131d6a83f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:39.099069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d0bd0ef1ee46fa9169513131d6a83f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:39.099782 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d0bd0ef1ee46fa9169513131d6a83f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:39.112984 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1346/2679] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:38 2022] DELETE /identity/v3/users/42d0bd0ef1ee46fa9169513131d6a83f => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:39.117490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:39.117993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:39.118091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:39.137767 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:39.138991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:39.198957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1334/2680] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:08:39.208455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:39.212789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:39.242446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:39.273828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dabf46-a998-4d23-9124-065f7c0c9ef0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:39.278594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72c268-6de6-4db2-94ca-294c6a7f77e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:39.279232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c268-6de6-4db2-94ca-294c6a7f77e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:39.279509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c268-6de6-4db2-94ca-294c6a7f77e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:39.279780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c268-6de6-4db2-94ca-294c6a7f77e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:39.289155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72c268-6de6-4db2-94ca-294c6a7f77e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:39.290579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72c268-6de6-4db2-94ca-294c6a7f77e3 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:39.300478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1347/2681] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:39.734199 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:39.736836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:39.750500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af75a96-a2da-4707-ada8-44f2d6cb4b1f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:39.755725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:39.756418 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:39.756727 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:39.757002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] PATH_INFO: `/v3/projects/22534fb887a943319c1ee0b1e9512de7` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:39.758161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22534fb887a943319c1ee0b1e9512de7)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:39.760092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:39.899100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22534fb887a943319c1ee0b1e9512de7', 'request_id': 'req-ddf45357-e2b0-4cf8-9254-8325c31da258'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:39.907854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22534fb887a943319c1ee0b1e9512de7', 'request_id': 'req-ddf45357-e2b0-4cf8-9254-8325c31da258'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:39.914585 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22534fb887a943319c1ee0b1e9512de7', 'request_id': 'req-ddf45357-e2b0-4cf8-9254-8325c31da258'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:40.001768 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] The token cache is being invalidate because project 22534fb887a943319c1ee0b1e9512de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:40.002156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:40.002845 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ddf45357-e2b0-4cf8-9254-8325c31da258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:40.004363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1335/2682] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:39 2022] DELETE /identity/v3/projects/22534fb887a943319c1ee0b1e9512de7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:40.009185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151871fa-3f7e-4603-8431-4821a3e0f652 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.009661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151871fa-3f7e-4603-8431-4821a3e0f652 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.009914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151871fa-3f7e-4603-8431-4821a3e0f652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.029687 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-151871fa-3f7e-4603-8431-4821a3e0f652 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:40.034763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-151871fa-3f7e-4603-8431-4821a3e0f652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:40.094145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1348/2683] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:40.101843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7251b-7840-461f-9a63-c49a03b7943a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.102353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7251b-7840-461f-9a63-c49a03b7943a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.102668 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7251b-7840-461f-9a63-c49a03b7943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.128414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f3d7251b-7840-461f-9a63-c49a03b7943a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:40.129795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f3d7251b-7840-461f-9a63-c49a03b7943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:40.138453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1336/2684] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:40.142969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151871fa-3f7e-4603-8431-4821a3e0f652 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:40.153239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151871fa-3f7e-4603-8431-4821a3e0f652 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:40.158323 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:40.161185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.161185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.161185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.161185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:08:40.161185 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:40.161958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:40.174030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1349/2685] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:08:40.178508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7251b-7840-461f-9a63-c49a03b7943a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:40.188865 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7251b-7840-461f-9a63-c49a03b7943a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:40.193605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:40.194190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.194470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.194765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.195394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:40.197276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:40.292763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:40.304420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1337/2686] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:08:40 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:40.305699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:40.317184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1025e6-7278-4788-b955-5d20a7804869 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:40.378703 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:40.379311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.379594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.379869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.380515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:40.382678 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:40.455950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1350/2687] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:08:40 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:40.458266 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:40.468743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c6533-efec-429e-9fc8-4a8734c55e65 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:40.473364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:40.473964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.474243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.474516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.475191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:40.476785 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:40.487466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1338/2688] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:40.491207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:40.503575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65e4fe-8a40-4abb-aee3-7544cca11500 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:40.508758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b0695-8b70-4c36-93c7-d73742859090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:40.509416 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b0695-8b70-4c36-93c7-d73742859090 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.509845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b0695-8b70-4c36-93c7-d73742859090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.510116 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b0695-8b70-4c36-93c7-d73742859090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.511336 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b0695-8b70-4c36-93c7-d73742859090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:40.513020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b0695-8b70-4c36-93c7-d73742859090 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:40.524435 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1351/2689] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:40.528627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:40.539497 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9372684-4fc3-4309-bfdf-b80c36b0ff63 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:40.545092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:40.545092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.545092 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.545562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] PATH_INFO: `/v3/projects/9edeaa37b9364d5398eaf317f05bdf15/users/29bbe9bf456b47b5bc3a31fb8028d000/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.560932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edeaa37b9364d5398eaf317f05bdf15, user_id=29bbe9bf456b47b5bc3a31fb8028d000, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:40.563907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:40.674380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1339/2690] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:40 2022] PUT /identity/v3/projects/9edeaa37b9364d5398eaf317f05bdf15/users/29bbe9bf456b47b5bc3a31fb8028d000/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:40.685600 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc5fbf-61b9-4cee-b3cf-0f1e37d72ae0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.685950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc5fbf-61b9-4cee-b3cf-0f1e37d72ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.686575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc5fbf-61b9-4cee-b3cf-0f1e37d72ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.715936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-27dc5fbf-61b9-4cee-b3cf-0f1e37d72ae0 None None] MFA Rules not processed for user `29bbe9bf456b47b5bc3a31fb8028d000`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:40.717148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-27dc5fbf-61b9-4cee-b3cf-0f1e37d72ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:40.747609 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:40.791589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1352/2691] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:40.803026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:40.804885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:40.864660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a934c49-4a1f-4a7d-aefa-24b772544361 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:40.868313 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098b051-c0f9-447b-87a8-6584d7c85bf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:40.868917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b051-c0f9-447b-87a8-6584d7c85bf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:40.869195 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b051-c0f9-447b-87a8-6584d7c85bf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:40.869496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098b051-c0f9-447b-87a8-6584d7c85bf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:40.878035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098b051-c0f9-447b-87a8-6584d7c85bf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:40.879504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098b051-c0f9-447b-87a8-6584d7c85bf3 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:40.889453 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1340/2692] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:44.998031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc5fbf-61b9-4cee-b3cf-0f1e37d72ae0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:45.059224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc5fbf-61b9-4cee-b3cf-0f1e37d72ae0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:45.063680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:45.064288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:45.064565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:45.064835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] PATH_INFO: `/v3/users/29bbe9bf456b47b5bc3a31fb8028d000` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:45.065931 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bbe9bf456b47b5bc3a31fb8028d000)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:45.067845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:45.265856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bbe9bf456b47b5bc3a31fb8028d000'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:45.266499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bbe9bf456b47b5bc3a31fb8028d000'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:45.360363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bbe9bf456b47b5bc3a31fb8028d000'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:45.372751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bbe9bf456b47b5bc3a31fb8028d000'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:45.373369 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bbe9bf456b47b5bc3a31fb8028d000'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:45.385211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1353/2693] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:44 2022] DELETE /identity/v3/users/29bbe9bf456b47b5bc3a31fb8028d000 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:45.390212 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:45.390714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:45.390847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:45.409806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:45.411044 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:45.471071 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1341/2694] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:45.480690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:45.482068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:45.528472 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:45.539510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca495d-0f51-4ea8-910f-45d8efd9041f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:45.543105 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d799f-8f97-448c-b25b-08c5ea2efaa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:45.543692 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d799f-8f97-448c-b25b-08c5ea2efaa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:45.543968 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d799f-8f97-448c-b25b-08c5ea2efaa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:45.544230 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d799f-8f97-448c-b25b-08c5ea2efaa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:45.552803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d799f-8f97-448c-b25b-08c5ea2efaa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:45.554302 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d799f-8f97-448c-b25b-08c5ea2efaa3 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:45.564173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1354/2695] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:45.967773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:45.969206 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:45.992672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f7c85-d9f8-4afe-8987-86226a8d9304 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:45.998139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:45.998740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:45.999024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:45.999321 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] PATH_INFO: `/v3/projects/9edeaa37b9364d5398eaf317f05bdf15` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.000633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9edeaa37b9364d5398eaf317f05bdf15)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:46.002600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:46.127736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9edeaa37b9364d5398eaf317f05bdf15', 'request_id': 'req-62b0645a-0517-41b3-9a63-64d43eb1d3e6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:46.134111 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9edeaa37b9364d5398eaf317f05bdf15', 'request_id': 'req-62b0645a-0517-41b3-9a63-64d43eb1d3e6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:46.141653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:46.142334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9edeaa37b9364d5398eaf317f05bdf15', 'request_id': 'req-62b0645a-0517-41b3-9a63-64d43eb1d3e6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:46.246240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] The token cache is being invalidate because project 9edeaa37b9364d5398eaf317f05bdf15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:46.246595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:46.247214 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-62b0645a-0517-41b3-9a63-64d43eb1d3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:46.248715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1342/2696] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:45 2022] DELETE /identity/v3/projects/9edeaa37b9364d5398eaf317f05bdf15 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:08:46.253473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb518b0-b44c-4465-b527-b6a86b9c61b9 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.253937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb518b0-b44c-4465-b527-b6a86b9c61b9 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.254025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb518b0-b44c-4465-b527-b6a86b9c61b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.274445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-6fb518b0-b44c-4465-b527-b6a86b9c61b9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:46.275718 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6fb518b0-b44c-4465-b527-b6a86b9c61b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:46.340163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1355/2697] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:46.348939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad21b7-c12d-4a37-b3be-3a7d9b9ccd0a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.349228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad21b7-c12d-4a37-b3be-3a7d9b9ccd0a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.349506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad21b7-c12d-4a37-b3be-3a7d9b9ccd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.370870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d2ad21b7-c12d-4a37-b3be-3a7d9b9ccd0a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:46.371086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d2ad21b7-c12d-4a37-b3be-3a7d9b9ccd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:46.378912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1343/2698] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:46.383347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb518b0-b44c-4465-b527-b6a86b9c61b9 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:46.394024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb518b0-b44c-4465-b527-b6a86b9c61b9 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:46.398615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:46.399348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.399621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.399887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.400551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:08:46.400846 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:46.402413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:46.414200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1356/2699] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:08:46.418380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad21b7-c12d-4a37-b3be-3a7d9b9ccd0a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:46.427653 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad21b7-c12d-4a37-b3be-3a7d9b9ccd0a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:46.432297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:46.433376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.433708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.434221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.435608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:46.437557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:46.550849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1344/2700] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:08:46 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:08:46.555244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:46.566078 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003dccc-194e-452a-be38-06a1ea0743f7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:46.623513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:46.624109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.624391 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.624665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.625275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:46.627211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:46.697756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1357/2701] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:08:46 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:08:46.703017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:46.713362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc15fb0-d469-436e-a90b-31ba91d7e51e admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:46.718084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:46.718696 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.718973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.719241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.719871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:46.721470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:46.731908 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1345/2702] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:46.735689 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:46.746181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa43c57-9754-4530-a1f5-82cbb111bf9d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:46.751068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1e390-9d6f-42d6-bbd3-9f4b24229c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:46.751653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1e390-9d6f-42d6-bbd3-9f4b24229c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.751933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1e390-9d6f-42d6-bbd3-9f4b24229c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.752199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1e390-9d6f-42d6-bbd3-9f4b24229c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.752824 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1e390-9d6f-42d6-bbd3-9f4b24229c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:46.754403 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1e390-9d6f-42d6-bbd3-9f4b24229c5c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:46.764667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1358/2703] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:08:46.768616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:46.778493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87205719-8436-4517-9360-0197b1442c78 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:46.783013 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:46.783664 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.783937 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.784210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] PATH_INFO: `/v3/projects/55d3f35793c44f40a2f60c60c05fbeba/users/57fe5ef22ae54ce48ea7b961f1d73a07/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.798763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d3f35793c44f40a2f60c60c05fbeba, user_id=57fe5ef22ae54ce48ea7b961f1d73a07, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:46.801347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:46.873859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1346/2704] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:46 2022] PUT /identity/v3/projects/55d3f35793c44f40a2f60c60c05fbeba/users/57fe5ef22ae54ce48ea7b961f1d73a07/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:46.884149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ff0af-ce89-4336-8ac7-3b6e741a4c9d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:46.886211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ff0af-ce89-4336-8ac7-3b6e741a4c9d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:46.886211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ff0af-ce89-4336-8ac7-3b6e741a4c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:46.911351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-549ff0af-ce89-4336-8ac7-3b6e741a4c9d None None] MFA Rules not processed for user `57fe5ef22ae54ce48ea7b961f1d73a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:46.912555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-549ff0af-ce89-4336-8ac7-3b6e741a4c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:46.953506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:46.976694 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1359/2705] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:46.981955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:47.046407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40ab5d-d70d-4c17-82cb-5f8e2acb0799 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:47.053228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:47.053228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.053228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.053228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.054131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:47.056840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:47.183604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1347/2706] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:08:46 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:47.187641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ff0af-ce89-4336-8ac7-3b6e741a4c9d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:47.198285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ff0af-ce89-4336-8ac7-3b6e741a4c9d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:47.254053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:47.254714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.254940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.255206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.255823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:47.257739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:47.365542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1360/2707] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:08:47 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:47.365730 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:47.369979 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f3b38-5111-43c0-ba85-7317afd59230 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:47.374662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:47.375376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.375654 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.375922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.376552 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:47.378131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:47.388763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1348/2708] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:47.392840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:47.403546 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54109bc5-5ba4-4770-b9d4-2943ba4bfd09 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:47.408126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cbb94-6192-4e96-b9e3-98ad7d0fb203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:47.408716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbb94-6192-4e96-b9e3-98ad7d0fb203 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.411601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbb94-6192-4e96-b9e3-98ad7d0fb203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.411601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbb94-6192-4e96-b9e3-98ad7d0fb203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.411601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cbb94-6192-4e96-b9e3-98ad7d0fb203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:47.411601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cbb94-6192-4e96-b9e3-98ad7d0fb203 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:47.426732 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1361/2709] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:08:47.428325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:47.438939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebeaf2-bcab-4414-a2e8-3d28fc9eff12 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:47.443534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:47.444169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.444447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.444717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] PATH_INFO: `/v3/projects/6ac346142f9f4b08ad587817c234aed3/users/eee98b66cb974d09b72f7e72078ed34e/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.458923 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac346142f9f4b08ad587817c234aed3, user_id=eee98b66cb974d09b72f7e72078ed34e, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:47.461575 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:47.526576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1349/2710] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:47 2022] PUT /identity/v3/projects/6ac346142f9f4b08ad587817c234aed3/users/eee98b66cb974d09b72f7e72078ed34e/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:47.538329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cfe12-d0fd-451e-8830-4834f4bdd329 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.538610 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cfe12-d0fd-451e-8830-4834f4bdd329 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.538911 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cfe12-d0fd-451e-8830-4834f4bdd329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.565758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3c1cfe12-d0fd-451e-8830-4834f4bdd329 None None] MFA Rules not processed for user `eee98b66cb974d09b72f7e72078ed34e`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:47.566941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3c1cfe12-d0fd-451e-8830-4834f4bdd329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:47.630107 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1362/2711] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:47.634457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:47.672655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:47.693681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a51e8-5cd6-47e9-8e76-d0917ac730bb admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:47.698229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:47.699377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.699755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.700118 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.700981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:47.703847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:47.802298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1350/2712] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:08:47 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:47.806342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cfe12-d0fd-451e-8830-4834f4bdd329 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:47.817119 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cfe12-d0fd-451e-8830-4834f4bdd329 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:47.861948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:47.873781 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:47.874377 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:47.874672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:47.874951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:47.875579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:47.877489 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:47.994519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1363/2713] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:08:47 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:08:47.999119 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:48.009218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcd37d-fd5a-4d55-92cd-9f6eaa780236 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:48.014364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:48.014982 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:48.015262 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:48.015534 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:48.016184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:48.017748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:48.027455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1351/2714] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:48.031615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:48.041681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b26b5d-5226-40c5-9858-15518f1f94de admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:48.046267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:48.046937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:48.047258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:48.047528 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] PATH_INFO: `/v3/projects/61b59e53aa2a4f5faa71f91156c483d4/users/014359955a9242daad8327afc23838c9/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:48.062492 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b59e53aa2a4f5faa71f91156c483d4, user_id=014359955a9242daad8327afc23838c9, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:48.065157 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:48.161207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1364/2715] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:48 2022] PUT /identity/v3/projects/61b59e53aa2a4f5faa71f91156c483d4/users/014359955a9242daad8327afc23838c9/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:48.165501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:48.175935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fca72-506a-4cff-be68-7341d3f4bf82 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:48.239112 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf159-29a3-45e8-b479-2b65659406bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:48.239709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf159-29a3-45e8-b479-2b65659406bb admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:48.239985 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf159-29a3-45e8-b479-2b65659406bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:48.240250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf159-29a3-45e8-b479-2b65659406bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:48.240876 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf159-29a3-45e8-b479-2b65659406bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:48.242458 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf159-29a3-45e8-b479-2b65659406bb admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:48.253037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1352/2716] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:08:48.257160 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:48.267708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14065303-c9dd-49c4-8922-851929b16fcb admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:48.272285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:48.272930 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:48.273206 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:48.273529 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] PATH_INFO: `/v3/projects/61b59e53aa2a4f5faa71f91156c483d4/users/014359955a9242daad8327afc23838c9/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:48.275461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b59e53aa2a4f5faa71f91156c483d4, user_id=014359955a9242daad8327afc23838c9, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:48.278043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:48.288616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1365/2717] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:08:48 2022] PUT /identity/v3/projects/61b59e53aa2a4f5faa71f91156c483d4/users/014359955a9242daad8327afc23838c9/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:08:48.299270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:48.299563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:48.299839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:48.329465 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] MFA Rules not processed for user `014359955a9242daad8327afc23838c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:48.330689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:48.399237 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1353/2718] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:48.418803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:48.420558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:48.481147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efd056-d923-4d23-8ec2-504db7930121 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:48.484793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:48.485440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:48.485722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:48.486608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:48.542390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:48.544214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:48.553045 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1366/2719] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:50.545477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:50.550717 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:50.558976 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c53ff8-432b-4638-b3b2-d3d19fe54af4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:50.562462 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:50.563228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:50.563503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:50.563772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:50.595650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:50.622802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:50.624292 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:50.633244 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1354/2720] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:51.983764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:51.992389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e5c7-e7de-4ca4-93a8-baec92c9d1f8 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:51.995965 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f88fdd-843d-4a2b-9a7b-74d11d57c85c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:51.996545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f88fdd-843d-4a2b-9a7b-74d11d57c85c service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:51.996823 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f88fdd-843d-4a2b-9a7b-74d11d57c85c service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:51.997099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f88fdd-843d-4a2b-9a7b-74d11d57c85c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:52.005821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f88fdd-843d-4a2b-9a7b-74d11d57c85c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:52.007259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f88fdd-843d-4a2b-9a7b-74d11d57c85c service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:52.016942 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1367/2721] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:56.665222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:56.727072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b338e-6c32-47d8-8d1d-dc0cf5be763c service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:56.732097 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:56.732715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:56.732991 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:56.733258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] PATH_INFO: `/v3/users/57fe5ef22ae54ce48ea7b961f1d73a07` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:56.734369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=57fe5ef22ae54ce48ea7b961f1d73a07)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:56.736297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:56.913860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57fe5ef22ae54ce48ea7b961f1d73a07'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:56.914510 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57fe5ef22ae54ce48ea7b961f1d73a07'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:56.915145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57fe5ef22ae54ce48ea7b961f1d73a07'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:56.922546 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57fe5ef22ae54ce48ea7b961f1d73a07'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:56.934030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57fe5ef22ae54ce48ea7b961f1d73a07'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:57.017810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1355/2722] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:56 2022] DELETE /identity/v3/users/57fe5ef22ae54ce48ea7b961f1d73a07 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:57.022219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:57.022709 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:57.022802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:57.041770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:08:57.042983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:57.079756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:57.100751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1368/2723] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:08:57.111396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:57.113151 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:57.165610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:57.169299 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a364c-3ea1-48db-b51d-cf904bee63aa admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:57.173542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:57.174163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:57.174415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:57.174714 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:57.183543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:57.184959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:57.194236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1356/2724] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:08:57.619365 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:57.624397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:57.638263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6839f7d-50d7-4707-a1fc-8d18fcdbb1b5 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:57.644843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:57.645508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:57.645771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:57.646056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] PATH_INFO: `/v3/projects/55d3f35793c44f40a2f60c60c05fbeba` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:57.647366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d3f35793c44f40a2f60c60c05fbeba)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:57.649300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:57.745958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d3f35793c44f40a2f60c60c05fbeba', 'request_id': 'req-6dd60046-cb5f-447c-96ab-cb6c86380b71'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:57.752216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d3f35793c44f40a2f60c60c05fbeba', 'request_id': 'req-6dd60046-cb5f-447c-96ab-cb6c86380b71'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:57.758697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d3f35793c44f40a2f60c60c05fbeba', 'request_id': 'req-6dd60046-cb5f-447c-96ab-cb6c86380b71'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:57.838200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] The token cache is being invalidate because project 55d3f35793c44f40a2f60c60c05fbeba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:57.838488 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:57.839142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:57.840624 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1369/2725] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:57 2022] DELETE /identity/v3/projects/55d3f35793c44f40a2f60c60c05fbeba => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:57.844680 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:57.854213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177c7d6-91f8-48f8-b944-6bb40bc1e781 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:57.909740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:57.910294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:57.910576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:57.910873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] PATH_INFO: `/v3/users/eee98b66cb974d09b72f7e72078ed34e` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:57.911980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee98b66cb974d09b72f7e72078ed34e)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:57.913885 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:58.085844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee98b66cb974d09b72f7e72078ed34e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.086535 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee98b66cb974d09b72f7e72078ed34e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.087155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee98b66cb974d09b72f7e72078ed34e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.093898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee98b66cb974d09b72f7e72078ed34e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.105669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee98b66cb974d09b72f7e72078ed34e'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.172332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1357/2726] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:57 2022] DELETE /identity/v3/users/eee98b66cb974d09b72f7e72078ed34e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:58.579677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:58.583744 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:58.595205 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60046-cb5f-447c-96ab-cb6c86380b71 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:58.652263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:58.652881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:58.653155 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:58.653423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] PATH_INFO: `/v3/projects/6ac346142f9f4b08ad587817c234aed3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:58.654549 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac346142f9f4b08ad587817c234aed3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:58.656958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:58.754014 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac346142f9f4b08ad587817c234aed3', 'request_id': 'req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.760594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac346142f9f4b08ad587817c234aed3', 'request_id': 'req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.767591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac346142f9f4b08ad587817c234aed3', 'request_id': 'req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.771310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:58.844476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] The token cache is being invalidate because project 6ac346142f9f4b08ad587817c234aed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:58.844766 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.845502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:58.847056 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1370/2727] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:58 2022] DELETE /identity/v3/projects/6ac346142f9f4b08ad587817c234aed3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:58.851154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:58.852585 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:58.865922 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6bf6-b6cf-4afe-9276-689411c0586b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:58.921707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:58.922356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:58.922639 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:58.922955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] PATH_INFO: `/v3/users/014359955a9242daad8327afc23838c9` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:58.924063 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=014359955a9242daad8327afc23838c9)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:58.925964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:58.938455 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:08:59.093541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014359955a9242daad8327afc23838c9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.094181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014359955a9242daad8327afc23838c9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.094804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014359955a9242daad8327afc23838c9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.101725 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014359955a9242daad8327afc23838c9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.115146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-ca93c6d8-929f-441f-bf8e-fa9a34e3f63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014359955a9242daad8327afc23838c9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.189042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1358/2728] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:08:58 2022] DELETE /identity/v3/users/014359955a9242daad8327afc23838c9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:08:59.595519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:08:59.596978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:08:59.610224 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b978bbe-a8c4-4b6c-91b1-1d370bda7238 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:08:59.667809 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:08:59.668419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:08:59.668693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:08:59.668961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] PATH_INFO: `/v3/projects/61b59e53aa2a4f5faa71f91156c483d4` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:08:59.670051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b59e53aa2a4f5faa71f91156c483d4)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:08:59.672518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:08:59.790334 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b59e53aa2a4f5faa71f91156c483d4', 'request_id': 'req-6b1db159-d718-4a98-9aac-f7b7156e8ad7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.796362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b59e53aa2a4f5faa71f91156c483d4', 'request_id': 'req-6b1db159-d718-4a98-9aac-f7b7156e8ad7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.802540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b59e53aa2a4f5faa71f91156c483d4', 'request_id': 'req-6b1db159-d718-4a98-9aac-f7b7156e8ad7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.880350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] The token cache is being invalidate because project 61b59e53aa2a4f5faa71f91156c483d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:08:59.880637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.881256 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:08:59.882769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1371/2729] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:08:59 2022] DELETE /identity/v3/projects/61b59e53aa2a4f5faa71f91156c483d4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:09:01.877164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aec622-ea2d-4eba-9520-5de319c9c7fc None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:01.877834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aec622-ea2d-4eba-9520-5de319c9c7fc None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:01.877834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aec622-ea2d-4eba-9520-5de319c9c7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:01.927673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-67aec622-ea2d-4eba-9520-5de319c9c7fc None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:01.928896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-67aec622-ea2d-4eba-9520-5de319c9c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:01.993220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1359/2730] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 13:09:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:02.120152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:02.121637 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:02.213218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1db159-d718-4a98-9aac-f7b7156e8ad7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:02.218909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:02.219510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:02.219786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:02.220055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:02.226023 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:02.228211 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:02.229771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:02.238575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1372/2731] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 13:09:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:09:11.990320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=62,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:12.309697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:19.342820 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:19.343265 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:19.343380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:19.361888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] MFA Rules not processed for user `44144b71637548c0b31368fc8114b6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:19.363386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:19.370354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1360/2732] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 13:09:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:19.382478 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:19.393629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe2641-d016-45be-9871-6d30e2544b34 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:19.398576 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '909708a2a44a48e7b62a6f6a8d792f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:19.399226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:19.399507 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:19.399789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:19.407543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:19.409097 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:19.417059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1373/2733] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 13:09:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:09:22.355169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:22.359685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:22.423428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5c493-54c6-42d2-a1f4-b8dfcf7b8d5e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:22.427996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:22.428750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:22.428895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:22.429169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] PATH_INFO: `/v3/users/7eb2f527351f49f4aef0be7535f60baa` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:22.430308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb2f527351f49f4aef0be7535f60baa)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:22.432276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:22.615712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb2f527351f49f4aef0be7535f60baa'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:22.616366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb2f527351f49f4aef0be7535f60baa'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:22.616993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb2f527351f49f4aef0be7535f60baa'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:22.623990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb2f527351f49f4aef0be7535f60baa'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:22.636385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-de0c39a6-15bf-4b0b-9010-7c4062b771a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb2f527351f49f4aef0be7535f60baa'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:22.708967 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1361/2734] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:09:22 2022] DELETE /identity/v3/users/7eb2f527351f49f4aef0be7535f60baa => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:09:22.940081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:22.951372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41729-b739-4776-aec9-802e35e85b09 service swift] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:23.012493 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:23.013106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.013378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.013649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] PATH_INFO: `/v3/projects/262256e7068b41fa96dde68614cfc126` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.014756 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=262256e7068b41fa96dde68614cfc126)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:23.016628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:23.116901 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '262256e7068b41fa96dde68614cfc126', 'request_id': 'req-c93980c2-d248-405f-bc3c-58c716892969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:23.122617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '262256e7068b41fa96dde68614cfc126', 'request_id': 'req-c93980c2-d248-405f-bc3c-58c716892969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:23.129121 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '262256e7068b41fa96dde68614cfc126', 'request_id': 'req-c93980c2-d248-405f-bc3c-58c716892969'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:23.206428 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] The token cache is being invalidate because project 262256e7068b41fa96dde68614cfc126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:09:23.206639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:23.207284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c93980c2-d248-405f-bc3c-58c716892969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:23.208763 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1374/2735] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:09:22 2022] DELETE /identity/v3/projects/262256e7068b41fa96dde68614cfc126 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:09:23.214574 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b91af-a2a1-4251-b95e-ac76f1a02506 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.214896 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b91af-a2a1-4251-b95e-ac76f1a02506 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.215171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b91af-a2a1-4251-b95e-ac76f1a02506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.234935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-d47b91af-a2a1-4251-b95e-ac76f1a02506 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:23.236778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d47b91af-a2a1-4251-b95e-ac76f1a02506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:23.293969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1362/2736] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:23.302262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25e0d4-5482-48c1-ada1-e77f33222d91 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.302544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25e0d4-5482-48c1-ada1-e77f33222d91 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.302858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25e0d4-5482-48c1-ada1-e77f33222d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.321834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ce25e0d4-5482-48c1-ada1-e77f33222d91 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:23.323053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ce25e0d4-5482-48c1-ada1-e77f33222d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:23.331261 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1375/2737] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:23.335315 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b91af-a2a1-4251-b95e-ac76f1a02506 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:23.344842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b91af-a2a1-4251-b95e-ac76f1a02506 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:23.349869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:23.350446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.350754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.351024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.351684 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:09:23.351980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:23.353562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:23.361881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:23.365127 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1363/2738] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:09:23.369346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25e0d4-5482-48c1-ada1-e77f33222d91 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:23.378348 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25e0d4-5482-48c1-ada1-e77f33222d91 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:23.382947 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:23.383523 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.383801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.384066 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.384672 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:23.386545 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:23.482414 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1376/2739] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:09:23 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:09:23.486837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:23.496992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d642e-edc2-427d-92d4-c17a29e1e3ec admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:23.553344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:23.553992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.554232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.554503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.555145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:23.557035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:23.656437 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1364/2740] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:09:23 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:09:23.665059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:23.676070 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19f31-b798-496f-b42d-cd9fd0edade7 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:23.680723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:23.683418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.683418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.683418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.683418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:23.684055 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:23.694407 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1377/2741] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:09:23.698460 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:23.709775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f332157-f016-4fe9-9466-7bdfc4f22ea2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:23.714873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f093ca-8723-441e-96de-f6f626e3a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:23.715474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f093ca-8723-441e-96de-f6f626e3a89e admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.715755 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f093ca-8723-441e-96de-f6f626e3a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.716025 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f093ca-8723-441e-96de-f6f626e3a89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.716658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f093ca-8723-441e-96de-f6f626e3a89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:23.718286 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f093ca-8723-441e-96de-f6f626e3a89e admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:23.729877 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1365/2742] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:09:23.734151 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:23.744893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e08a40-db08-41df-ae9b-c4b639c21fbe admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:23.749379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:23.750018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.750295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.750565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] PATH_INFO: `/v3/projects/2ff6d74cc3514170be74023a189a41c1/users/2e3f8e38f96746fba8bd2563e26672f3/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.765199 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff6d74cc3514170be74023a189a41c1, user_id=2e3f8e38f96746fba8bd2563e26672f3, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:23.767870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:23.839952 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1378/2743] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:09:23 2022] PUT /identity/v3/projects/2ff6d74cc3514170be74023a189a41c1/users/2e3f8e38f96746fba8bd2563e26672f3/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:09:23.851549 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9a7aa-52e0-4645-9d12-e617a1034aee None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:23.851837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9a7aa-52e0-4645-9d12-e617a1034aee None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:23.852110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9a7aa-52e0-4645-9d12-e617a1034aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:23.879795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-fdd9a7aa-52e0-4645-9d12-e617a1034aee None None] MFA Rules not processed for user `2e3f8e38f96746fba8bd2563e26672f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:23.881002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-fdd9a7aa-52e0-4645-9d12-e617a1034aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:23.943496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1366/2744] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:23.953967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:23.955656 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:23.969661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:24.012419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9079a39-44b4-45f0-80cd-5127144a6699 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:24.015994 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1c84e-0235-464f-b767-0d79454f5e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:24.016590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c84e-0235-464f-b767-0d79454f5e7f service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:24.016899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c84e-0235-464f-b767-0d79454f5e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:24.017170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c84e-0235-464f-b767-0d79454f5e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:24.025288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1c84e-0235-464f-b767-0d79454f5e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:24.026731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1c84e-0235-464f-b767-0d79454f5e7f service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:24.036305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1379/2745] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:09:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:09:33.940996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:34.068197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=20 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:35.146473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9a7aa-52e0-4645-9d12-e617a1034aee None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:35.205015 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9a7aa-52e0-4645-9d12-e617a1034aee None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:35.210159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:35.210838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:35.211122 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:35.211397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] PATH_INFO: `/v3/users/2e3f8e38f96746fba8bd2563e26672f3` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:35.212544 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3f8e38f96746fba8bd2563e26672f3)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:35.214436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:35.396058 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3f8e38f96746fba8bd2563e26672f3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:35.396616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3f8e38f96746fba8bd2563e26672f3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:35.397210 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3f8e38f96746fba8bd2563e26672f3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:35.404177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3f8e38f96746fba8bd2563e26672f3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:35.419340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3f8e38f96746fba8bd2563e26672f3'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:35.497824 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1367/2746] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:09:35 2022] DELETE /identity/v3/users/2e3f8e38f96746fba8bd2563e26672f3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:09:35.502406 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:35.502933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:35.503037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:35.531458 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:35.532639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:35.588859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1380/2747] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:35.599305 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:35.600881 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:35.666706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920851a-9302-4166-8109-9f898e1860dc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:35.670250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf60b7-56c3-4d02-abcf-8f362ba01072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:35.670871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf60b7-56c3-4d02-abcf-8f362ba01072 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:35.671153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf60b7-56c3-4d02-abcf-8f362ba01072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:35.671428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf60b7-56c3-4d02-abcf-8f362ba01072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:35.680278 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf60b7-56c3-4d02-abcf-8f362ba01072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:35.681704 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf60b7-56c3-4d02-abcf-8f362ba01072 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:35.691302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1368/2748] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:09:36.071789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.073209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:36.085748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3260458-91d5-42d2-8657-e3ba4488b214 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.090405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.091196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.091625 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.091815 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] PATH_INFO: `/v3/projects/2ff6d74cc3514170be74023a189a41c1` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.092949 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff6d74cc3514170be74023a189a41c1)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.094887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:36.183827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff6d74cc3514170be74023a189a41c1', 'request_id': 'req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:36.189248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff6d74cc3514170be74023a189a41c1', 'request_id': 'req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:36.195997 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff6d74cc3514170be74023a189a41c1', 'request_id': 'req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:36.240390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] The token cache is being invalidate because project 2ff6d74cc3514170be74023a189a41c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:09:36.240677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:36.241292 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:09:36.242803 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1381/2749] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:09:36 2022] DELETE /identity/v3/projects/2ff6d74cc3514170be74023a189a41c1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:09:36.247969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d400-cbf7-4fe6-96ad-6d5ab229c29b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.248431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d400-cbf7-4fe6-96ad-6d5ab229c29b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.248539 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d400-cbf7-4fe6-96ad-6d5ab229c29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.268363 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9812d400-cbf7-4fe6-96ad-6d5ab229c29b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:36.269553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9812d400-cbf7-4fe6-96ad-6d5ab229c29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:36.277784 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:36.315433 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.317381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:36.332614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1369/2750] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:36.340531 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938bf4c-381b-4a25-b595-fd372bfc6bb9 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.340805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938bf4c-381b-4a25-b595-fd372bfc6bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.341080 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938bf4c-381b-4a25-b595-fd372bfc6bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.408838 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:36.419527 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e9bb-4bac-4758-a6c9-e3c8ced4aa66 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.425074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.425704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.426555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.426853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.428826 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:36.441176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.442845 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:36.454264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1382/2751] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:09:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:09:36.485353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a938bf4c-381b-4a25-b595-fd372bfc6bb9 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:36.486556 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a938bf4c-381b-4a25-b595-fd372bfc6bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:36.495353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1370/2752] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:09:36.499723 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.510442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fd5d5-4b9c-4707-92f3-3ef1405e1934 service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.515440 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.518223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.518223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.518223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.518223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:09:36.518223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.519419 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:36.531124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1383/2753] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:09:36.536143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938bf4c-381b-4a25-b595-fd372bfc6bb9 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.545318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938bf4c-381b-4a25-b595-fd372bfc6bb9 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.549870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.550451 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.551398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.551674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.552288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.554161 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:36.651622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1371/2754] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:09:36 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:09:36.652699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.663657 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2831a-d795-4f9c-ba9b-0086e0a9da5a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.719519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.720044 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.720326 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.720598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.721221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.723675 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:36.823753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1384/2755] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:09:36 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:09:36.827853 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.841311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989babe-6297-43c5-b064-ef6c4863088c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.845888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.846471 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.846780 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.847054 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.848297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.849890 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:36.860632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1372/2756] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:09:36.865059 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.875042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce5c19-e607-4717-a0b6-fe148c941969 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.879757 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f5e5d-d939-4584-bdb0-068acbce8f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.880422 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f5e5d-d939-4584-bdb0-068acbce8f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.880769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f5e5d-d939-4584-bdb0-068acbce8f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.881108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f5e5d-d939-4584-bdb0-068acbce8f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.881807 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f5e5d-d939-4584-bdb0-068acbce8f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.883508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f5e5d-d939-4584-bdb0-068acbce8f0b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:36.895411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1385/2757] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:09:36.899516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:36.909596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154800c-8352-4511-bf0e-8c58d3e3ea32 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:36.914746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:36.915381 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:36.915660 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:36.915930 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] PATH_INFO: `/v3/projects/618334fc8ef14486a1fc1411c42bf9b7/users/5b6dec9001a14679ade64276a7962b5f/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:36.929832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618334fc8ef14486a1fc1411c42bf9b7, user_id=5b6dec9001a14679ade64276a7962b5f, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:36.932480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:37.018108 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1373/2758] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:09:36 2022] PUT /identity/v3/projects/618334fc8ef14486a1fc1411c42bf9b7/users/5b6dec9001a14679ade64276a7962b5f/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:09:37.024343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902c8b-0869-4f5f-b4e5-a983c4149d82 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:37.024613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902c8b-0869-4f5f-b4e5-a983c4149d82 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:37.024884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902c8b-0869-4f5f-b4e5-a983c4149d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:37.045022 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-af902c8b-0869-4f5f-b4e5-a983c4149d82 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:37.046213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-af902c8b-0869-4f5f-b4e5-a983c4149d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:37.104025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1386/2759] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:37.113783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:37.115495 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:37.171609 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39063f63-fa61-4583-9612-4d944f7c43f3 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:37.175221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:37.176000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:37.176282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:37.176555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:37.185338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:37.186783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:37.196596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1374/2760] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:09:42.579962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d34c1-defd-478b-8af8-57561ef25a46 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:42.580486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d34c1-defd-478b-8af8-57561ef25a46 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:42.580486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d34c1-defd-478b-8af8-57561ef25a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:42.611979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e10d34c1-defd-478b-8af8-57561ef25a46 None None] MFA Rules not processed for user `5b6dec9001a14679ade64276a7962b5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:42.614225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e10d34c1-defd-478b-8af8-57561ef25a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:42.657822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:42.683480 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1387/2761] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:42.688128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:42.698163 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269ff8-8489-4878-8637-b766acd9837f service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:42.702858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:42.703454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:42.703731 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:42.703999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:42.704620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:42.706557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:42.796576 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1375/2762] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:09:42 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:09:42.800784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d34c1-defd-478b-8af8-57561ef25a46 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:42.811629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d34c1-defd-478b-8af8-57561ef25a46 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:42.881405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:42.882043 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:42.882324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:42.882595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:42.883263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:42.885218 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:42.961213 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1388/2763] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:09:42 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:09:42.966804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:42.977386 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1a492-cca1-41c5-9f4a-03002f722b6f admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:42.983261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:42.983895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:42.984171 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:42.984443 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:42.985082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:42.986835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:42.999616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1376/2764] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:09:43.004623 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:43.015346 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b281595-462e-45e8-9010-c64ceca54406 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:43.021606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:43.022298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:43.022676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:43.023614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] PATH_INFO: `/v3/projects/4e1b2fa5ec714b5f93e67dd70d81246f/users/1b6acbc630c04c76aa9f73cb3d779edf/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:43.042626 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1b2fa5ec714b5f93e67dd70d81246f, user_id=1b6acbc630c04c76aa9f73cb3d779edf, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:43.045397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:43.145108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1389/2765] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:09:43 2022] PUT /identity/v3/projects/4e1b2fa5ec714b5f93e67dd70d81246f/users/1b6acbc630c04c76aa9f73cb3d779edf/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:09:43.149164 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:43.159546 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00feeb-fca4-4073-846e-252de0ede44b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:43.177384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:43.230454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58edfba-1399-438b-8421-b94e3498d875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:43.230454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58edfba-1399-438b-8421-b94e3498d875 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:43.230454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58edfba-1399-438b-8421-b94e3498d875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:43.230454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58edfba-1399-438b-8421-b94e3498d875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:43.231052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58edfba-1399-438b-8421-b94e3498d875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:43.231052 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58edfba-1399-438b-8421-b94e3498d875 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:43.241046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1377/2766] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:09:43.245388 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:43.270682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa06e2-0d51-4610-9656-627592d2914c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:43.270682 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:43.271208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:43.271587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:43.271961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] PATH_INFO: `/v3/projects/4e1b2fa5ec714b5f93e67dd70d81246f/users/1b6acbc630c04c76aa9f73cb3d779edf/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:43.274738 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1b2fa5ec714b5f93e67dd70d81246f, user_id=1b6acbc630c04c76aa9f73cb3d779edf, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:43.277508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:43.290054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1390/2767] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:09:43 2022] PUT /identity/v3/projects/4e1b2fa5ec714b5f93e67dd70d81246f/users/1b6acbc630c04c76aa9f73cb3d779edf/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:09:48.498064 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b7332-85ed-4e47-a752-cb46f18d9f03 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:48.498529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b7332-85ed-4e47-a752-cb46f18d9f03 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:48.498603 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b7332-85ed-4e47-a752-cb46f18d9f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:48.533801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b04b7332-85ed-4e47-a752-cb46f18d9f03 None None] MFA Rules not processed for user `1b6acbc630c04c76aa9f73cb3d779edf`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:48.539082 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b04b7332-85ed-4e47-a752-cb46f18d9f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:48.617371 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1378/2768] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:48.632778 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:09:48.634640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:48.715008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715c2af-0d72-44b3-a830-44300910f907 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:09:48.720513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:09:48.726353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:48.726353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:48.726353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:48.754879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:09:48.782467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:09:48.784135 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:09:48.794143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1391/2769] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:09:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:09:51.295920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:09:51.296349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:09:51.296466 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:09:51.315425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] MFA Rules not processed for user `5b6dec9001a14679ade64276a7962b5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:09:51.316643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:09:51.324558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1379/2770] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:09:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:09:58.828083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=14 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:10:01.310072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=57 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:10:33.164497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:10:33.227265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7c0c-24fc-4fb2-8352-dff53b01819f service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:10:33.231486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:10:33.232083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:10:33.232361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:10:33.232629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:10:33.262235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:10:33.263702 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:10:33.273884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1392/2771] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:10:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:10:43.270299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=40 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:11:30.006026 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:11:30.007522 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:11:30.036917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eccb6-d172-4716-b5cc-db5dd9301aa2 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:11:30.040433 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:11:30.041024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:11:30.041297 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:11:30.041563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:11:30.043302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:11:30.052463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:11:30.053854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:11:30.061636 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1380/2772] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:11:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:11:40.058154 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=12 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:12:10.105724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:12:10.107134 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:12:10.167615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27af48-1b1c-4b52-89d2-6252cc1e7062 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:12:10.171122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:12:10.171705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:12:10.171978 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:12:10.172243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:12:10.173917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:12:10.242877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:12:10.244321 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:12:10.253225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1393/2773] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:12:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:12:20.249038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=53 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:12:50.275583 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:12:50.284545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdbf0e-0331-488a-a740-d3beb8d931ff service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:12:50.288349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:12:50.289594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:12:50.289594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:12:50.289594 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:12:50.291995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:12:50.305168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:12:50.307273 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:12:50.318306 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1381/2774] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:12:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:12:51.514150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:12:51.523473 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b3190-583a-46d7-8138-9f03b096b80c service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:12:51.528427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:12:51.528427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:12:51.528427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:12:51.528427 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:12:51.529940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:12:51.543300 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:12:51.544750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:12:51.555571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1394/2775] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:12:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:00.313340 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:01.550714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:05.978558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:05.987859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb4107-bb37-42e7-9903-6d7a9b61f506 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:05.991557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:05.992156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:05.992700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:05.992969 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:05.994766 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:06.004904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:06.006323 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:06.014605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1382/2776] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:13:06.106817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:06.115579 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c653e1-0c1c-4657-8ff2-7d9d7e2f2486 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:06.119530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:06.120700 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:06.121094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:06.121367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:06.123127 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:06.137172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:06.138521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:06.149253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1395/2777] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:06.215051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:06.216431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:06.290553 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a52cd7-4be3-4836-b141-8cc85bf48c72 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:06.296141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:06.296735 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:06.297009 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:06.297276 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:06.298987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:06.309919 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:06.311508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:06.320966 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1383/2778] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:16.144602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:16.316912 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=38 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:19.038870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:19.047969 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8a6a1-84a5-4737-b9af-dca006d29812 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:19.051662 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:19.052247 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:19.052524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:19.052798 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:19.054751 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:19.129330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:19.131035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:19.141318 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1396/2779] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:23.030863 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:23.041884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa8489-aa9b-4c8b-9a48-dbcc47a3b315 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:23.047211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:23.048000 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:23.048349 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:23.048681 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:23.050592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:23.063605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:23.065301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:23.076441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1384/2780] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:23.158210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:23.161725 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:23.173252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdb992-eb09-4a95-8ff7-a3d64ce12366 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:23.177801 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:23.178437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:23.178758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:23.179031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] PATH_INFO: `/v3/users/751f05f1ff6c4593870073d8f47e0db9` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:23.180196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=751f05f1ff6c4593870073d8f47e0db9)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:23.183042 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:23.363178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751f05f1ff6c4593870073d8f47e0db9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.363750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751f05f1ff6c4593870073d8f47e0db9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.435363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751f05f1ff6c4593870073d8f47e0db9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.448779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751f05f1ff6c4593870073d8f47e0db9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.449747 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-ec652a29-35cc-4993-84f9-89dee888adb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751f05f1ff6c4593870073d8f47e0db9'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.462490 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1397/2781] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:13:23 2022] DELETE /identity/v3/users/751f05f1ff6c4593870073d8f47e0db9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:13:23.698732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:23.710393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac110b-d085-44aa-bd63-3358d67992bf service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:23.770842 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37366444-6876-423d-be89-066e695174d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:23.771473 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37366444-6876-423d-be89-066e695174d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:23.771747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37366444-6876-423d-be89-066e695174d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:23.772019 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37366444-6876-423d-be89-066e695174d2 admin admin] PATH_INFO: `/v3/projects/e7a3075a9e3a47a4bcb9136ce8303173` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:23.773157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37366444-6876-423d-be89-066e695174d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a3075a9e3a47a4bcb9136ce8303173)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:23.775078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37366444-6876-423d-be89-066e695174d2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:23.904732 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-37366444-6876-423d-be89-066e695174d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a3075a9e3a47a4bcb9136ce8303173', 'request_id': 'req-37366444-6876-423d-be89-066e695174d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.910709 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-37366444-6876-423d-be89-066e695174d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a3075a9e3a47a4bcb9136ce8303173', 'request_id': 'req-37366444-6876-423d-be89-066e695174d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.916382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-37366444-6876-423d-be89-066e695174d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a3075a9e3a47a4bcb9136ce8303173', 'request_id': 'req-37366444-6876-423d-be89-066e695174d2'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.988799 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-37366444-6876-423d-be89-066e695174d2 admin admin] The token cache is being invalidate because project e7a3075a9e3a47a4bcb9136ce8303173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:13:23.989086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-37366444-6876-423d-be89-066e695174d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.989718 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-37366444-6876-423d-be89-066e695174d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:13:23.991391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1385/2782] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:13:23 2022] DELETE /identity/v3/projects/e7a3075a9e3a47a4bcb9136ce8303173 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:13:23.997448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870da75-bb5a-4947-af00-65513faf2852 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:23.997802 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870da75-bb5a-4947-af00-65513faf2852 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:23.998084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870da75-bb5a-4947-af00-65513faf2852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.018086 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b870da75-bb5a-4947-af00-65513faf2852 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:13:24.019310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b870da75-bb5a-4947-af00-65513faf2852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:24.076929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1398/2783] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:13:24.085944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722fca-d953-44a0-afe7-2886c73f36ed None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.086420 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722fca-d953-44a0-afe7-2886c73f36ed None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.086523 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722fca-d953-44a0-afe7-2886c73f36ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.105952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-97722fca-d953-44a0-afe7-2886c73f36ed None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:13:24.107181 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-97722fca-d953-44a0-afe7-2886c73f36ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:24.114900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1386/2784] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:13:24.119281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870da75-bb5a-4947-af00-65513faf2852 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:24.128714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870da75-bb5a-4947-af00-65513faf2852 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:24.133276 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:24.133861 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.134138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.134408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.135074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:13:24.135873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:24.137461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:24.138926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:24.149310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1399/2785] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:13:24.154144 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97722fca-d953-44a0-afe7-2886c73f36ed None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:24.163251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97722fca-d953-44a0-afe7-2886c73f36ed None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:24.169017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:24.169608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.169884 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.170150 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.170800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:24.172724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:24.293417 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1387/2786] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:13:24 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:13:24.297670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:24.307411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b27608-0238-470b-a8b8-06615aef06af admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:24.364181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:24.364772 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.365051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.365322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.365941 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:24.368136 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:24.486893 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1400/2787] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:13:24 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:13:24.491131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:24.502983 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7804efb-16a0-4660-b21a-6765258f30d2 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:24.507509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:24.508094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.508394 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.508667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.509328 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:24.510948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:24.521440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1388/2788] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:13:24.525351 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:24.535118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057210d-e14e-4775-a9b9-9d5d2dfe5651 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:24.539693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f13b6-d5c2-4dde-ba30-c86e2f0aa7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:24.540277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f13b6-d5c2-4dde-ba30-c86e2f0aa7bc admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.540558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f13b6-d5c2-4dde-ba30-c86e2f0aa7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.540833 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f13b6-d5c2-4dde-ba30-c86e2f0aa7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.541460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f13b6-d5c2-4dde-ba30-c86e2f0aa7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:24.543613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f13b6-d5c2-4dde-ba30-c86e2f0aa7bc admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:24.554111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1401/2789] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:13:24.558792 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:24.568642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c248f-ae1a-4582-b9d1-be0286e87bb8 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:24.573282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:24.577936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.577936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.577936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] PATH_INFO: `/v3/projects/5e9455e7426b490680c92f1bb3cecca0/users/fa9364936c614a68b22f66c32b76cba9/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.588469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9455e7426b490680c92f1bb3cecca0, user_id=fa9364936c614a68b22f66c32b76cba9, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:24.591100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:24.657645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1389/2790] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:13:24 2022] PUT /identity/v3/projects/5e9455e7426b490680c92f1bb3cecca0/users/fa9364936c614a68b22f66c32b76cba9/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:13:24.662736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160399-0dfb-4a98-8eef-12f02409bb0b None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.663024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160399-0dfb-4a98-8eef-12f02409bb0b None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.663988 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160399-0dfb-4a98-8eef-12f02409bb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.682172 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-f6160399-0dfb-4a98-8eef-12f02409bb0b None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:13:24.683923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f6160399-0dfb-4a98-8eef-12f02409bb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:24.741937 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1402/2791] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:13:24.752586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:24.754341 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:24.768165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:24.810103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da659945-e23e-4922-bc2a-9e9da59d5433 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:24.813688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:24.814280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:24.814558 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:24.814874 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:24.824130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:24.830096 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:24.834754 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1390/2792] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:29.602068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ab430-bae7-4377-8e16-fc476690aedf None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:29.602548 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ab430-bae7-4377-8e16-fc476690aedf None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:29.602644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ab430-bae7-4377-8e16-fc476690aedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:29.633446 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-111ab430-bae7-4377-8e16-fc476690aedf None None] MFA Rules not processed for user `fa9364936c614a68b22f66c32b76cba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:13:29.634903 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-111ab430-bae7-4377-8e16-fc476690aedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:29.709159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1403/2793] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:13:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:13:29.724382 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:29.726178 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:29.804061 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285885fa-5f2e-4ee0-92a8-d9f145e56538 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:29.809563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:29.810176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:29.810448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:29.810743 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:29.819204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:29.820790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:29.831281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1391/2794] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:31.939180 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087b25-c485-40f9-8def-99a2935be0b0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:31.939677 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087b25-c485-40f9-8def-99a2935be0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:31.939793 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087b25-c485-40f9-8def-99a2935be0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:31.946736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:31.960728 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-b0087b25-c485-40f9-8def-99a2935be0b0 None None] MFA Rules not processed for user `fa9364936c614a68b22f66c32b76cba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:13:31.962131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b0087b25-c485-40f9-8def-99a2935be0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:31.969264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1404/2795] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:13:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:13:32.309043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:32.310490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:32.322633 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823632-24c7-43dc-98ae-60f383b74daf service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:32.326284 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:32.326933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:32.327217 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:32.327490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:32.329251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:32.339629 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:32.341081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:32.349980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1392/2796] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:36.865804 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0087b25-c485-40f9-8def-99a2935be0b0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:36.880038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0087b25-c485-40f9-8def-99a2935be0b0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:36.885614 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:36.886219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:36.891338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:36.891338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:36.891338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:36.904617 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:36.906208 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:36.917784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1405/2797] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:42.345873 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=60 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:43.456446 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:13:43.457996 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:43.474927 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca22e1-8df2-4671-aba0-e576bc998e84 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:13:43.480649 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:13:43.481467 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:13:43.481747 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:13:43.482020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:13:43.483765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:13:43.496616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:13:43.498213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:13:43.509899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1393/2798] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:13:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:13:46.912530 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=8 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:13:53.504500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:14:45.308307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:14:45.309726 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:14:45.379385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23d173-0ce6-4041-89ba-4b0958051d33 service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:14:45.384443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:14:45.385176 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:14:45.385459 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:14:45.385724 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:14:45.387517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:14:45.399474 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:14:45.401343 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:14:45.411092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1406/2799] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:14:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:14:51.676147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:14:51.685536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e6005-58d8-404f-a168-b6f72b64abe1 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:14:51.689107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:14:51.689686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:14:51.689963 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:14:51.690232 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:14:51.691978 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:14:51.749396 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:14:51.750860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:14:51.759949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1394/2800] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:14:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:14:55.407025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=32 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:14:58.277533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:14:58.287643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd1377-b562-412a-b1d0-d0b20d0d0653 service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:14:58.291193 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:14:58.291784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:14:58.292068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:14:58.292339 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:14:58.294062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:14:58.305885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:14:58.307383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:14:58.318381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1407/2801] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:15:01.755692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=28 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:15:05.056412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:15:05.069559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2d5e0-ff29-420b-a08a-0dde3c27722d service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:15:05.080450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:15:05.082169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:05.082169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:05.082169 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:05.084159 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:05.096862 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:15:05.099361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:15:05.109434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1395/2802] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:15:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:15:05.164437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:15:05.166041 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:05.183219 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46a88e-b770-47c8-8fef-e97c716e10ff service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:15:05.189286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:15:05.190007 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:05.190288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:05.190559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] PATH_INFO: `/v3/users/5b6dec9001a14679ade64276a7962b5f` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:05.191704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6dec9001a14679ade64276a7962b5f)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:15:05.193853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:15:05.384939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6dec9001a14679ade64276a7962b5f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:05.385588 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6dec9001a14679ade64276a7962b5f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:05.501698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6dec9001a14679ade64276a7962b5f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:05.511253 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6dec9001a14679ade64276a7962b5f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:05.511954 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6dec9001a14679ade64276a7962b5f'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:05.528400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1408/2803] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:15:05 2022] DELETE /identity/v3/users/5b6dec9001a14679ade64276a7962b5f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:15:05.756408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:15:05.767431 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cf031-4f5b-4072-97c0-b39fd97c09ff service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:15:05.823893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:15:05.824509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:05.824783 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:05.825050 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] PATH_INFO: `/v3/projects/618334fc8ef14486a1fc1411c42bf9b7` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:05.826686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618334fc8ef14486a1fc1411c42bf9b7)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:15:05.828597 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:15:05.937346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618334fc8ef14486a1fc1411c42bf9b7', 'request_id': 'req-580bee36-4306-41ef-bf40-8c3393ca7942'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:05.943414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618334fc8ef14486a1fc1411c42bf9b7', 'request_id': 'req-580bee36-4306-41ef-bf40-8c3393ca7942'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:05.948907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618334fc8ef14486a1fc1411c42bf9b7', 'request_id': 'req-580bee36-4306-41ef-bf40-8c3393ca7942'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.039905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] The token cache is being invalidate because project 618334fc8ef14486a1fc1411c42bf9b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:15:06.040101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.040711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.042179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1396/2804] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:15:05 2022] DELETE /identity/v3/projects/618334fc8ef14486a1fc1411c42bf9b7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:15:06.046164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:15:06.047595 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:06.060962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7b231-cb85-4256-9834-4097a7b64ae8 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:15:06.115970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:15:06.116591 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:06.116870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:06.117139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] PATH_INFO: `/v3/users/1b6acbc630c04c76aa9f73cb3d779edf` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:06.118255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b6acbc630c04c76aa9f73cb3d779edf)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:15:06.120510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:15:06.309938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b6acbc630c04c76aa9f73cb3d779edf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.311081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b6acbc630c04c76aa9f73cb3d779edf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.376958 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b6acbc630c04c76aa9f73cb3d779edf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.382293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:15:06.389959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b6acbc630c04c76aa9f73cb3d779edf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.390735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-c26227da-42ad-4236-aeff-92f2d6c48cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b6acbc630c04c76aa9f73cb3d779edf'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.402604 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1409/2805] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:15:06 2022] DELETE /identity/v3/users/1b6acbc630c04c76aa9f73cb3d779edf => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:15:06.641957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:15:06.643482 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:06.656503 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bee36-4306-41ef-bf40-8c3393ca7942 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:15:06.712475 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:15:06.713100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:06.713376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:06.713645 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] PATH_INFO: `/v3/projects/4e1b2fa5ec714b5f93e67dd70d81246f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:06.714776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1b2fa5ec714b5f93e67dd70d81246f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:15:06.716702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:15:06.736871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:15:06.817779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1b2fa5ec714b5f93e67dd70d81246f', 'request_id': 'req-727a7409-0217-4826-ab04-8246f073cbfc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.823724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1b2fa5ec714b5f93e67dd70d81246f', 'request_id': 'req-727a7409-0217-4826-ab04-8246f073cbfc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.829078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1b2fa5ec714b5f93e67dd70d81246f', 'request_id': 'req-727a7409-0217-4826-ab04-8246f073cbfc'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.902425 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] The token cache is being invalidate because project 4e1b2fa5ec714b5f93e67dd70d81246f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:15:06.902744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.903368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:15:06.905353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1397/2806] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:15:06 2022] DELETE /identity/v3/projects/4e1b2fa5ec714b5f93e67dd70d81246f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:15:14.183923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:14.184408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:14.184501 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:14.203542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] MFA Rules not processed for user `fa9364936c614a68b22f66c32b76cba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:15:14.204740 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:14.258137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1410/2807] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:15:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:15:14.586569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:15:14.588156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:14.644608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a7409-0217-4826-ab04-8246f073cbfc admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:15:14.648133 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:15:14.648724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:14.649001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:14.649270 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:14.650990 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:14.660186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:15:14.661590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:15:14.669411 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1398/2808] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:15:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:15:24.251278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=34,keystone:DELETE=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:15:24.665902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:15:28.042098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:15:28.043584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:28.119266 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878c5fc-dd91-45f8-a03a-41283bf2b372 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:15:28.125594 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc203a-6a37-4cf8-8eec-3fdcd776c9d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:15:28.126197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc203a-6a37-4cf8-8eec-3fdcd776c9d6 service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:15:28.126477 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc203a-6a37-4cf8-8eec-3fdcd776c9d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:15:28.126771 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc203a-6a37-4cf8-8eec-3fdcd776c9d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:15:28.128487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-b6fc203a-6a37-4cf8-8eec-3fdcd776c9d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:15:28.141465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc203a-6a37-4cf8-8eec-3fdcd776c9d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:15:28.143117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc203a-6a37-4cf8-8eec-3fdcd776c9d6 service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:15:28.153760 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1411/2809] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:15:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:15:38.149131 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=34 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:16:36.036203 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:16:36.037667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:16:36.060338 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5db48-19eb-4c24-81b9-b311e00ee5f0 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:16:36.063851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:16:36.064436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:16:36.064706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:16:36.064972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:16:36.066599 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:16:36.075547 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:16:36.076952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:16:36.085584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1399/2810] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:16:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:16:46.082205 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=11 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:16:59.430382 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:16:59.430834 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:16:59.430972 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:16:59.449552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] MFA Rules not processed for user `fa9364936c614a68b22f66c32b76cba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:16:59.450783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:16:59.457553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1412/2811] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:16:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:17:09.444308 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:21.503470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:21.504834 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:21.579157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11fa1b-aadf-4b72-86dd-b023452baa41 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:21.582641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:21.583246 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.583516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.583781 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.586302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:21.610037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:21.611459 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:21.622180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1400/2812] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:18:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:18:29.418670 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.420074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:29.432587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554448-6bca-4870-8505-cde9ae6a08a1 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.436154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7b51f-dccc-4ae2-beee-017dc55ef945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.436752 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7b51f-dccc-4ae2-beee-017dc55ef945 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.437031 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7b51f-dccc-4ae2-beee-017dc55ef945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.437302 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7b51f-dccc-4ae2-beee-017dc55ef945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.439024 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-21b7b51f-dccc-4ae2-beee-017dc55ef945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:29.450062 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7b51f-dccc-4ae2-beee-017dc55ef945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.451518 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7b51f-dccc-4ae2-beee-017dc55ef945 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.460221 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1413/2813] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:18:29.874468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.883152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf13a3-3e27-48e6-ac4c-57cd5cf49853 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.886686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.887271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.887550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.887816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.889496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:29.908801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.910211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.931311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1401/2814] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:18:39.456223 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.488441 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.488813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.489051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.508761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] MFA Rules not processed for user `fa9364936c614a68b22f66c32b76cba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:39.509982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:39.516639 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1414/2815] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:18:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:18:39.814774 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.823832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2749a4-20a4-4a1c-8d09-7ac1b002e0d7 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.827308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.827892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.828165 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.828434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.830189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:39.839879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.841293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.849155 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1402/2816] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:18:40.254299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.256307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:40.324967 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2b4e5-1f31-45c3-aceb-5783bc258937 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.329902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.330498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.330814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.331087 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.332825 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:40.344764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.346330 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.356352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1415/2817] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:18:43.703366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:43.704862 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:43.721179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3f05f-7b3c-4e29-8427-74e0c675f6de service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:43.727657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:43.734739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:43.734739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:43.734739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:43.734739 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:43.744215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:43.745791 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:43.756760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1403/2818] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:18:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:18:48.974027 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.986106 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0c15f-cb1e-4b77-951d-39fab76bbb5c service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.992190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4093c8-c71d-4dcc-bc26-147c6ecd7a0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.992817 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4093c8-c71d-4dcc-bc26-147c6ecd7a0c service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.993098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4093c8-c71d-4dcc-bc26-147c6ecd7a0c service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.993367 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4093c8-c71d-4dcc-bc26-147c6ecd7a0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.995173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-1a4093c8-c71d-4dcc-bc26-147c6ecd7a0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:49.008345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4093c8-c71d-4dcc-bc26-147c6ecd7a0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.009939 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4093c8-c71d-4dcc-bc26-147c6ecd7a0c service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.020606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1416/2819] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:18:53.752346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=65 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:59.016016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=41 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:50.278137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.279545 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:50.331972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925d004-1743-4616-af9f-aeff3fd6d88d service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.337491 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.338086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.338366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.338641 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.340380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:50.354939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:50.354939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:50.364461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1404/2820] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:00.360191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=23 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:29.701643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf94ce-e2db-463c-a6e9-295695d05d9d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:29.702074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf94ce-e2db-463c-a6e9-295695d05d9d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:29.702183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf94ce-e2db-463c-a6e9-295695d05d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:29.760713 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-88bf94ce-e2db-463c-a6e9-295695d05d9d None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:29.761910 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-88bf94ce-e2db-463c-a6e9-295695d05d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:29.768265 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1417/2821] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 24 13:20:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:29.774711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:29.784037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0c1e-26eb-4bfb-b5d4-35d536729fb7 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:29.787601 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:29.788186 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:29.788463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:29.788733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:29.790441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:29.872100 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:29.873703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:29.892687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1405/2822] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:33.340051 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf94ce-e2db-463c-a6e9-295695d05d9d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:33.381368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf94ce-e2db-463c-a6e9-295695d05d9d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:33.386476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:33.387113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:33.387395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:33.387679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:33.389401 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:33.400790 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:33.403691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:33.412297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1418/2823] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:33.471608 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:33.472999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:33.507814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c702b5e-afb4-4896-9c11-a1c6effd77e3 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:33.512141 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:33.512750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:33.513024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:33.513294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] PATH_INFO: `/v3/users/fa9364936c614a68b22f66c32b76cba9` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:33.514378 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9364936c614a68b22f66c32b76cba9)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:33.516385 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:33.706711 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9364936c614a68b22f66c32b76cba9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:33.707359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9364936c614a68b22f66c32b76cba9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:33.707965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9364936c614a68b22f66c32b76cba9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:33.715180 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9364936c614a68b22f66c32b76cba9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:33.727369 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4929c454-9a13-4331-b148-2de503561611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9364936c614a68b22f66c32b76cba9'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:33.813905 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1406/2824] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:20:33 2022] DELETE /identity/v3/users/fa9364936c614a68b22f66c32b76cba9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:34.061608 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:34.072245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da66cf5-3ad9-4a66-8979-e25130b110da service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:34.128310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:34.128938 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.129220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.129487 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] PATH_INFO: `/v3/projects/5e9455e7426b490680c92f1bb3cecca0` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.130667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9455e7426b490680c92f1bb3cecca0)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:34.132571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:34.236645 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9455e7426b490680c92f1bb3cecca0', 'request_id': 'req-b311a8d8-be70-4b77-9c8c-6f936cbafe01'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:34.242750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9455e7426b490680c92f1bb3cecca0', 'request_id': 'req-b311a8d8-be70-4b77-9c8c-6f936cbafe01'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:34.249216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9455e7426b490680c92f1bb3cecca0', 'request_id': 'req-b311a8d8-be70-4b77-9c8c-6f936cbafe01'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:34.299178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] The token cache is being invalidate because project 5e9455e7426b490680c92f1bb3cecca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:20:34.299461 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:34.300178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-b311a8d8-be70-4b77-9c8c-6f936cbafe01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:34.301686 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1419/2825] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:20:34 2022] DELETE /identity/v3/projects/5e9455e7426b490680c92f1bb3cecca0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:34.307345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aec2a-addc-4418-8f17-1c3357871a78 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.307571 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aec2a-addc-4418-8f17-1c3357871a78 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.307844 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aec2a-addc-4418-8f17-1c3357871a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.334745 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:34.354758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-f38aec2a-addc-4418-8f17-1c3357871a78 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:34.355992 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-f38aec2a-addc-4418-8f17-1c3357871a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:34.413899 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1407/2826] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:34.423372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82be21-f9de-4f04-b084-7adef075864e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.423894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82be21-f9de-4f04-b084-7adef075864e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.423894 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82be21-f9de-4f04-b084-7adef075864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.443767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7f82be21-f9de-4f04-b084-7adef075864e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:34.444962 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7f82be21-f9de-4f04-b084-7adef075864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:34.452737 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1420/2827] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:34.456861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aec2a-addc-4418-8f17-1c3357871a78 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:34.467204 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aec2a-addc-4418-8f17-1c3357871a78 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:34.471761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:34.472334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.472607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.472875 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.473514 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:34.473810 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:34.475432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:34.487327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1408/2828] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:34.491372 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82be21-f9de-4f04-b084-7adef075864e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:34.500618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82be21-f9de-4f04-b084-7adef075864e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:34.505200 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:34.505814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.506091 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.506362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.507020 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:34.508932 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:34.513103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:34.580667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1421/2829] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:20:34 2022] POST /identity/v3/projects => generated 403 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:34.584771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:34.595279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb4b19-1f98-463b-956b-a6bd5f4f082b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:34.649972 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:34.650555 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.650858 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.651128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.651742 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:34.653625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:34.728288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1409/2830] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:20:34 2022] POST /identity/v3/users => generated 302 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:20:34.732396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:34.742586 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dcac8-7f06-49eb-b9b0-a15e60ebd0e5 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:34.747170 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:34.748028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.748303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.748571 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.749195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:34.750806 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:34.760633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1422/2831] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:34.764329 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:34.775215 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506409a-8ae2-4fd5-a435-1cdd4d3e35eb admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:34.779827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1b59c-e63d-470a-b527-e2ec746ac2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:34.780414 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1b59c-e63d-470a-b527-e2ec746ac2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.780695 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1b59c-e63d-470a-b527-e2ec746ac2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.780970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1b59c-e63d-470a-b527-e2ec746ac2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.781607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1b59c-e63d-470a-b527-e2ec746ac2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:34.783254 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1b59c-e63d-470a-b527-e2ec746ac2aa admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:34.795149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1410/2832] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:20:34.799030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:34.808714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bf659-7210-45a4-92eb-6d930521b329 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:34.813241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:34.813870 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.814145 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.814415 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] PATH_INFO: `/v3/projects/0bb2ce5652cd4d8fb71f28a1274c8e69/users/d4d5b6ecebfa4eaba21e1d83d04ca3c7/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.829786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb2ce5652cd4d8fb71f28a1274c8e69, user_id=d4d5b6ecebfa4eaba21e1d83d04ca3c7, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:34.832589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:34.899902 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1423/2833] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:34 2022] PUT /identity/v3/projects/0bb2ce5652cd4d8fb71f28a1274c8e69/users/d4d5b6ecebfa4eaba21e1d83d04ca3c7/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:34.910010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa58ec-b70a-4b8e-8638-f3af2e25205a None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:34.910295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa58ec-b70a-4b8e-8638-f3af2e25205a None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:34.910569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa58ec-b70a-4b8e-8638-f3af2e25205a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:34.938086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-c3fa58ec-b70a-4b8e-8638-f3af2e25205a None None] MFA Rules not processed for user `d4d5b6ecebfa4eaba21e1d83d04ca3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:34.939320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-c3fa58ec-b70a-4b8e-8638-f3af2e25205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:34.984542 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:35.001093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1411/2834] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:35.005683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:35.066532 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47801f-5711-40a4-9e14-a5c4d70619bf admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:35.071239 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:35.071829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.072111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.072385 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.072998 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:35.074920 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:35.187816 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1424/2835] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:20:35 2022] POST /identity/v3/projects => generated 403 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:35.191949 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa58ec-b70a-4b8e-8638-f3af2e25205a None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:35.203182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa58ec-b70a-4b8e-8638-f3af2e25205a None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:35.258964 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:35.259564 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.259839 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.260109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.260715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:35.262619 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:35.342255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1412/2836] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:20:35 2022] POST /identity/v3/users => generated 308 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:20:35.346324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:35.357789 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f323-7de2-45a6-8789-5e9d710e6e78 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:35.362750 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:35.368035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.368035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.368035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.368035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:35.368035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:35.382181 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1425/2837] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:20:35.383469 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:35.393762 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26494018-724f-4f04-92a5-b0abc7e91c03 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:35.399600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:35.400291 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.400566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.400835 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] PATH_INFO: `/v3/projects/48de6f2590d8485dbcc907bbdd6ac1eb/users/01d8027c0340428f83ab5f7637efa4b6/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.418398 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48de6f2590d8485dbcc907bbdd6ac1eb, user_id=01d8027c0340428f83ab5f7637efa4b6, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:35.421256 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:35.501303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1413/2838] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:35 2022] PUT /identity/v3/projects/48de6f2590d8485dbcc907bbdd6ac1eb/users/01d8027c0340428f83ab5f7637efa4b6/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:35.505366 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:35.515922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7758b2-722e-49a7-8df8-5d1f81a8a7d0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:35.571225 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a42ddc-21d2-4e7d-9336-e8f0f8e0cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:35.571810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a42ddc-21d2-4e7d-9336-e8f0f8e0cf13 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.572088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a42ddc-21d2-4e7d-9336-e8f0f8e0cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.572354 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a42ddc-21d2-4e7d-9336-e8f0f8e0cf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.572976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a42ddc-21d2-4e7d-9336-e8f0f8e0cf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:35.574551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a42ddc-21d2-4e7d-9336-e8f0f8e0cf13 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:35.584049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1426/2839] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:35.588182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:35.598316 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c10f-004e-4bcf-9015-8fa2f1310e58 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:35.603422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:35.604048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.604322 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.604591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] PATH_INFO: `/v3/projects/48de6f2590d8485dbcc907bbdd6ac1eb/users/01d8027c0340428f83ab5f7637efa4b6/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.606470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48de6f2590d8485dbcc907bbdd6ac1eb, user_id=01d8027c0340428f83ab5f7637efa4b6, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:35.609088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:35.619432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1414/2840] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:35 2022] PUT /identity/v3/projects/48de6f2590d8485dbcc907bbdd6ac1eb/users/01d8027c0340428f83ab5f7637efa4b6/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:35.630854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61045dc-08de-43cf-970a-1bb6f21477ba None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.631137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61045dc-08de-43cf-970a-1bb6f21477ba None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.631409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61045dc-08de-43cf-970a-1bb6f21477ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.651267 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:35.658919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-d61045dc-08de-43cf-970a-1bb6f21477ba None None] MFA Rules not processed for user `01d8027c0340428f83ab5f7637efa4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:35.660286 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d61045dc-08de-43cf-970a-1bb6f21477ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:35.732236 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1427/2841] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:35 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:35.743625 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:35.745268 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:35.801320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f33705-dce7-47ec-99aa-257171812565 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:35.805535 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5ba81-644d-4e60-bfb3-6f353f03aeeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:35.806131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ba81-644d-4e60-bfb3-6f353f03aeeb service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.806406 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ba81-644d-4e60-bfb3-6f353f03aeeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.806703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ba81-644d-4e60-bfb3-6f353f03aeeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.815605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5ba81-644d-4e60-bfb3-6f353f03aeeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:35.817020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5ba81-644d-4e60-bfb3-6f353f03aeeb service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:35.826269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1415/2842] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:35 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:35.973774 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61045dc-08de-43cf-970a-1bb6f21477ba None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:36.035418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61045dc-08de-43cf-970a-1bb6f21477ba None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:36.040761 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:36.041380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.041663 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.041933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] PATH_INFO: `/v3/users/d4d5b6ecebfa4eaba21e1d83d04ca3c7` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.043084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d5b6ecebfa4eaba21e1d83d04ca3c7)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:36.044981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:36.234035 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d5b6ecebfa4eaba21e1d83d04ca3c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:36.234722 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d5b6ecebfa4eaba21e1d83d04ca3c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:36.299830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d5b6ecebfa4eaba21e1d83d04ca3c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:36.312748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d5b6ecebfa4eaba21e1d83d04ca3c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:36.313396 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f97551db-e958-4d83-bacf-54214b185976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d5b6ecebfa4eaba21e1d83d04ca3c7'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:36.326186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1428/2843] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:20:35 2022] DELETE /identity/v3/users/d4d5b6ecebfa4eaba21e1d83d04ca3c7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:36.341763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927d478-32ec-430c-a907-e370d183f167 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.342068 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927d478-32ec-430c-a907-e370d183f167 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.342419 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927d478-32ec-430c-a907-e370d183f167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.361084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-9927d478-32ec-430c-a907-e370d183f167 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:36.362288 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9927d478-32ec-430c-a907-e370d183f167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:36.402508 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:36.419984 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1416/2844] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:36.429296 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.430238 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.430539 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.449697 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:36.450929 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:36.460212 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:36.502540 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1429/2845] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 24 13:20:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:36.508389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927d478-32ec-430c-a907-e370d183f167 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:36.517808 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927d478-32ec-430c-a907-e370d183f167 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:36.521428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:36.522024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.522344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.522621 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.531577 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:36.532995 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:36.542957 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1417/2846] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:36.896037 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:36.897542 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:36.912164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c238608-25e0-4755-9c95-fab0ebc50975 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:36.917245 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:36.917868 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.918143 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.918413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] PATH_INFO: `/v3/projects/0bb2ce5652cd4d8fb71f28a1274c8e69` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.919560 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb2ce5652cd4d8fb71f28a1274c8e69)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:36.921465 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:37.014114 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb2ce5652cd4d8fb71f28a1274c8e69', 'request_id': 'req-4586f01b-8e2f-4309-9648-0c516eefaec1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.019613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb2ce5652cd4d8fb71f28a1274c8e69', 'request_id': 'req-4586f01b-8e2f-4309-9648-0c516eefaec1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.026363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb2ce5652cd4d8fb71f28a1274c8e69', 'request_id': 'req-4586f01b-8e2f-4309-9648-0c516eefaec1'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.098780 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] The token cache is being invalidate because project 0bb2ce5652cd4d8fb71f28a1274c8e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:20:37.099054 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.099660 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.101138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1430/2847] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:20:36 2022] DELETE /identity/v3/projects/0bb2ce5652cd4d8fb71f28a1274c8e69 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:37.105693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:37.115452 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8232d8-95d3-4acb-9782-ef9fc6e72c73 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:37.170676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:37.171285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:37.171560 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:37.171826 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] PATH_INFO: `/v3/users/01d8027c0340428f83ab5f7637efa4b6` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:37.172911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d8027c0340428f83ab5f7637efa4b6)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:37.174821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:37.330928 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d8027c0340428f83ab5f7637efa4b6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.331670 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d8027c0340428f83ab5f7637efa4b6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.332277 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d8027c0340428f83ab5f7637efa4b6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.339939 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d8027c0340428f83ab5f7637efa4b6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.351870 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-bf0dfe2d-1ea9-41a4-8726-2d4d90e05644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d8027c0340428f83ab5f7637efa4b6'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:37.425637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1418/2848] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:20:37 2022] DELETE /identity/v3/users/01d8027c0340428f83ab5f7637efa4b6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:20:37.804246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:37.806303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:37.821118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f01b-8e2f-4309-9648-0c516eefaec1 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:37.878277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:37.878852 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:37.879130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:37.879397 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] PATH_INFO: `/v3/projects/48de6f2590d8485dbcc907bbdd6ac1eb` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:37.880499 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48de6f2590d8485dbcc907bbdd6ac1eb)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:37.882359 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.003371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48de6f2590d8485dbcc907bbdd6ac1eb', 'request_id': 'req-a775c51b-f8e4-41a8-a162-b3e38d2b4021'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:38.004558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:38.009775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48de6f2590d8485dbcc907bbdd6ac1eb', 'request_id': 'req-a775c51b-f8e4-41a8-a162-b3e38d2b4021'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:38.015821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48de6f2590d8485dbcc907bbdd6ac1eb', 'request_id': 'req-a775c51b-f8e4-41a8-a162-b3e38d2b4021'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:38.071148 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] The token cache is being invalidate because project 48de6f2590d8485dbcc907bbdd6ac1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:20:38.071462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:38.072083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-a775c51b-f8e4-41a8-a162-b3e38d2b4021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:38.073569 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1431/2849] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:20:37 2022] DELETE /identity/v3/projects/48de6f2590d8485dbcc907bbdd6ac1eb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:38.078428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685845fd-2142-4e8d-b9d3-808b1c37e4bf None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.078943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685845fd-2142-4e8d-b9d3-808b1c37e4bf None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.079042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685845fd-2142-4e8d-b9d3-808b1c37e4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.098703 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-685845fd-2142-4e8d-b9d3-808b1c37e4bf None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:38.101819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-685845fd-2142-4e8d-b9d3-808b1c37e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:38.145934 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:38.158140 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1419/2850] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:20:38.166390 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca88090-1510-4aa9-aa7b-a94a1ad382fb None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.166696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca88090-1510-4aa9-aa7b-a94a1ad382fb None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.166974 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca88090-1510-4aa9-aa7b-a94a1ad382fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.185977 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-3ca88090-1510-4aa9-aa7b-a94a1ad382fb None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:38.187196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3ca88090-1510-4aa9-aa7b-a94a1ad382fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:38.194912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1432/2851] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:38.199438 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685845fd-2142-4e8d-b9d3-808b1c37e4bf None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:38.209422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685845fd-2142-4e8d-b9d3-808b1c37e4bf None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:38.214626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:38.215235 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.215509 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.215772 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.216402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:38.216697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:38.218272 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.229665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1420/2852] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:38.233989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca88090-1510-4aa9-aa7b-a94a1ad382fb None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:38.243734 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca88090-1510-4aa9-aa7b-a94a1ad382fb None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:38.248767 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:38.249371 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.249628 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.249897 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.250510 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:38.252500 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.318255 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1433/2853] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:20:38 2022] POST /identity/v3/projects => generated 405 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:38.323251 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:38.333693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8e98d-3acc-4fe3-aec8-303d0de68752 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:38.388042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:38.388627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.388898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.389162 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.389776 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:38.391697 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.466672 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1421/2854] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:20:38 2022] POST /identity/v3/users => generated 303 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:38.471522 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:38.485163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0015d4b-4941-4e39-b0ce-4a532ac6260a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:38.489863 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:38.490464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.490785 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.491053 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.491693 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:38.493906 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.504443 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1434/2855] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:38.509106 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:38.519043 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2baf375-2695-4a67-9f60-3ff305a2c249 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:38.523578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d26975-8eba-4546-a525-2b841114c83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:38.524200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d26975-8eba-4546-a525-2b841114c83c admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.524474 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d26975-8eba-4546-a525-2b841114c83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.524736 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d26975-8eba-4546-a525-2b841114c83c admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.525359 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d26975-8eba-4546-a525-2b841114c83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:38.526994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d26975-8eba-4546-a525-2b841114c83c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.537128 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1422/2856] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:38.541004 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:38.550922 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9c8c-b038-4fe8-9072-b410f0cee261 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:38.556008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:38.556631 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.556909 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.557184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] PATH_INFO: `/v3/projects/67241319addd46c4b2a9d5c7185a6424/users/5811ae0889c04d3bb7458930544db5b2/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.572262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=67241319addd46c4b2a9d5c7185a6424, user_id=5811ae0889c04d3bb7458930544db5b2, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:38.574898 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.642676 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1435/2857] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:38 2022] PUT /identity/v3/projects/67241319addd46c4b2a9d5c7185a6424/users/5811ae0889c04d3bb7458930544db5b2/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:38.647661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e77ce5-533c-4a97-9997-51a75858bfc3 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.647944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e77ce5-533c-4a97-9997-51a75858bfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.648211 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e77ce5-533c-4a97-9997-51a75858bfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.666903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-39e77ce5-533c-4a97-9997-51a75858bfc3 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:38.668074 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-39e77ce5-533c-4a97-9997-51a75858bfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:38.723332 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1423/2858] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:38.733353 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:38.735034 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:38.794885 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de96909-cff8-47f2-902b-0ca7113bf7ed admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:38.798450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:38.799065 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:38.799344 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:38.799615 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:38.808982 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:38.810531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:38.820038 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1436/2859] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:20:39.883946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8fc6-5629-400d-96cf-3735f00ce20b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:39.884454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8fc6-5629-400d-96cf-3735f00ce20b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:39.884550 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8fc6-5629-400d-96cf-3735f00ce20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:39.912472 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-456f8fc6-5629-400d-96cf-3735f00ce20b None None] MFA Rules not processed for user `5811ae0889c04d3bb7458930544db5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:39.913685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-456f8fc6-5629-400d-96cf-3735f00ce20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:39.962213 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:39.979650 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1424/2860] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:39 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:39.984303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:39.995197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41862d01-38bd-4e0e-95cf-9605a22d80fc service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.000052 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.000621 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.000896 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.001163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.001805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.003753 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.112905 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1437/2861] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:20:39 2022] POST /identity/v3/projects => generated 405 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:20:40.117033 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8fc6-5629-400d-96cf-3735f00ce20b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.127893 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8fc6-5629-400d-96cf-3735f00ce20b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.182568 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.183190 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.183477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.183724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.184345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.186221 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.282567 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1425/2862] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:20:40 2022] POST /identity/v3/users => generated 309 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:40.286633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.296878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805d48a-f637-4b7a-80c1-9aba7df260c0 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.301386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.301970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.302248 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.302519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.303189 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.304769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.316280 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1438/2863] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:40.320804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.330763 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7bef2-d3a2-409c-a912-e442702a514d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.335238 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.336530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.336806 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.337072 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] PATH_INFO: `/v3/projects/2c1e046e27fa4db386dbb8e0c0bfede0/users/c3adf8908d6a41338324c0e3e2917942/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.351124 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1e046e27fa4db386dbb8e0c0bfede0, user_id=c3adf8908d6a41338324c0e3e2917942, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.353756 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.443146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1426/2864] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:40 2022] PUT /identity/v3/projects/2c1e046e27fa4db386dbb8e0c0bfede0/users/c3adf8908d6a41338324c0e3e2917942/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:40.447577 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.457476 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5570e-5e96-4bf2-a7dc-00ea5d666bbe admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.498712 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:40.513565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbe6ca-4a7b-4b08-bf01-6ff6211e0f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.514158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbe6ca-4a7b-4b08-bf01-6ff6211e0f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.514439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbe6ca-4a7b-4b08-bf01-6ff6211e0f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.514739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbe6ca-4a7b-4b08-bf01-6ff6211e0f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.515378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbe6ca-4a7b-4b08-bf01-6ff6211e0f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.516957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbe6ca-4a7b-4b08-bf01-6ff6211e0f68 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.526324 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1439/2865] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:40.530529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.540872 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7fe25-d1ee-455e-85c9-1cfad712ca3d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.545384 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.546020 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.546298 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.546674 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] PATH_INFO: `/v3/projects/2c1e046e27fa4db386dbb8e0c0bfede0/users/c3adf8908d6a41338324c0e3e2917942/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.548582 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1e046e27fa4db386dbb8e0c0bfede0, user_id=c3adf8908d6a41338324c0e3e2917942, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.551222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.562008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1427/2866] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:40 2022] PUT /identity/v3/projects/2c1e046e27fa4db386dbb8e0c0bfede0/users/c3adf8908d6a41338324c0e3e2917942/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:41.713079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ed70-c1d7-40ae-8051-5dea9b81673d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:41.713612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ed70-c1d7-40ae-8051-5dea9b81673d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:41.713612 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ed70-c1d7-40ae-8051-5dea9b81673d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:41.742887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2950ed70-c1d7-40ae-8051-5dea9b81673d None None] MFA Rules not processed for user `c3adf8908d6a41338324c0e3e2917942`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:41.744117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2950ed70-c1d7-40ae-8051-5dea9b81673d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:41.817304 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1440/2867] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:20:42.331078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:42.332974 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:42.405403 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c239be-95cb-4362-9026-e9d905401760 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:42.410456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:42.411085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:42.411362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:42.411631 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:42.413352 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:42.425120 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:42.426750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:42.436943 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1428/2868] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:43.135380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950ed70-c1d7-40ae-8051-5dea9b81673d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.198559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950ed70-c1d7-40ae-8051-5dea9b81673d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.203090 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.203704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.206976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.206976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] PATH_INFO: `/v3/users/5811ae0889c04d3bb7458930544db5b2` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.206976 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5811ae0889c04d3bb7458930544db5b2)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:43.207395 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:43.410981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5811ae0889c04d3bb7458930544db5b2'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.411556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5811ae0889c04d3bb7458930544db5b2'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.465973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5811ae0889c04d3bb7458930544db5b2'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.476400 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:43.478627 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5811ae0889c04d3bb7458930544db5b2'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.479281 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5811ae0889c04d3bb7458930544db5b2'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.491269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1441/2869] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:20:43 2022] DELETE /identity/v3/users/5811ae0889c04d3bb7458930544db5b2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:43.713168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.724421 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11123a-d888-43c2-8e59-4ed57a190f40 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.753895 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:43.781708 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.782319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.782595 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.782894 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] PATH_INFO: `/v3/projects/67241319addd46c4b2a9d5c7185a6424` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.783980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] RBAC: Authorizing `identity:delete_project(project_id=67241319addd46c4b2a9d5c7185a6424)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:43.785879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:43.865854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67241319addd46c4b2a9d5c7185a6424', 'request_id': 'req-3ad5034b-9c56-493a-896b-db25b4b0608d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.872365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67241319addd46c4b2a9d5c7185a6424', 'request_id': 'req-3ad5034b-9c56-493a-896b-db25b4b0608d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.878046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67241319addd46c4b2a9d5c7185a6424', 'request_id': 'req-3ad5034b-9c56-493a-896b-db25b4b0608d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.927626 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] The token cache is being invalidate because project 67241319addd46c4b2a9d5c7185a6424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:20:43.927911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.928528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:43.930012 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1429/2870] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:20:43 2022] DELETE /identity/v3/projects/67241319addd46c4b2a9d5c7185a6424 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:43.934029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.935462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:43.949342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbcaff-810a-4a4e-aa8c-ef9d67dc5c9f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:44.005263 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:44.005887 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:44.006166 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:44.006439 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] PATH_INFO: `/v3/users/c3adf8908d6a41338324c0e3e2917942` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:44.008237 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3adf8908d6a41338324c0e3e2917942)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:44.010150 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:44.141021 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3adf8908d6a41338324c0e3e2917942'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.141684 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3adf8908d6a41338324c0e3e2917942'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.201669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3adf8908d6a41338324c0e3e2917942'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.214405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3adf8908d6a41338324c0e3e2917942'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.216943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e8942d83-af71-4b82-b6ca-eef408cb9f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3adf8908d6a41338324c0e3e2917942'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.227117 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1442/2871] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:20:43 2022] DELETE /identity/v3/users/c3adf8908d6a41338324c0e3e2917942 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:44.450775 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:44.452280 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:44.466085 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5034b-9c56-493a-896b-db25b4b0608d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:44.523368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:44.523917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:44.524194 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:44.524463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] PATH_INFO: `/v3/projects/2c1e046e27fa4db386dbb8e0c0bfede0` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:44.525569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1e046e27fa4db386dbb8e0c0bfede0)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:44.527490 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:44.631366 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1e046e27fa4db386dbb8e0c0bfede0', 'request_id': 'req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.637294 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1e046e27fa4db386dbb8e0c0bfede0', 'request_id': 'req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.642707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1e046e27fa4db386dbb8e0c0bfede0', 'request_id': 'req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.644339 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:44.733413 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] The token cache is being invalidate because project 2c1e046e27fa4db386dbb8e0c0bfede0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:20:44.733997 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.734318 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-1c1ac059-944a-42a0-bdbf-06492f9eb9e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:20:44.735827 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1430/2872] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:20:44 2022] DELETE /identity/v3/projects/2c1e046e27fa4db386dbb8e0c0bfede0 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:44.740739 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647376fe-5123-4774-bb87-1652bd8b3f40 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:44.741036 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647376fe-5123-4774-bb87-1652bd8b3f40 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:44.741310 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647376fe-5123-4774-bb87-1652bd8b3f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:44.761297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-647376fe-5123-4774-bb87-1652bd8b3f40 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:44.762497 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-647376fe-5123-4774-bb87-1652bd8b3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:44.826029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1443/2873] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:44.833721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29236c-7616-4460-a297-46b51dd96a24 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:44.834004 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29236c-7616-4460-a297-46b51dd96a24 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:44.834279 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29236c-7616-4460-a297-46b51dd96a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:44.854024 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5d29236c-7616-4460-a297-46b51dd96a24 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:44.855250 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5d29236c-7616-4460-a297-46b51dd96a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:44.863035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1431/2874] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:44.867285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647376fe-5123-4774-bb87-1652bd8b3f40 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:44.874305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:44.878130 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647376fe-5123-4774-bb87-1652bd8b3f40 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:44.882759 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:44.883358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:44.886345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:44.886345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:44.886345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:44.886345 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:44.886727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:44.897636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1444/2875] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:44.902008 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29236c-7616-4460-a297-46b51dd96a24 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:44.917308 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29236c-7616-4460-a297-46b51dd96a24 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:44.922814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:44.923410 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:44.923689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:44.923960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:44.924600 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:44.926487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.032533 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1432/2876] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:20:44 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:45.036456 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.047175 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ccc71-0698-401f-81a1-80af7a7c4c7e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.101561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.102748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.103033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.103303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.103919 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.105814 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.215387 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1445/2877] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:20:45 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:20:45.219486 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.230409 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8199e-d7a5-4159-a3da-9445c886fe49 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.234981 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.235562 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.235883 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.236153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.236778 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.238345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.248689 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1433/2878] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:45.252475 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.262680 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56630e0-c054-441d-abf2-1810a6162c9d admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.267409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293677ef-cef1-4483-9be4-1660e94b2c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.268000 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293677ef-cef1-4483-9be4-1660e94b2c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.268283 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293677ef-cef1-4483-9be4-1660e94b2c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.268557 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293677ef-cef1-4483-9be4-1660e94b2c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.269232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293677ef-cef1-4483-9be4-1660e94b2c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.270951 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293677ef-cef1-4483-9be4-1660e94b2c6d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.281960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1446/2879] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:45.286022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.296692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117e99c-6f04-4c70-b57a-dcacc9be5a39 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.301264 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.301891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.302166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.302436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] PATH_INFO: `/v3/projects/ef22d0c9df444e05a90135ea0c91bf3e/users/d45fd67692aa4a3e88c0929b0a7de9ab/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.316361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef22d0c9df444e05a90135ea0c91bf3e, user_id=d45fd67692aa4a3e88c0929b0a7de9ab, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.319016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.391692 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1434/2880] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:45 2022] PUT /identity/v3/projects/ef22d0c9df444e05a90135ea0c91bf3e/users/d45fd67692aa4a3e88c0929b0a7de9ab/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:45.402455 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87f2b-6853-4e52-b0e3-153e85bb1814 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.402775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87f2b-6853-4e52-b0e3-153e85bb1814 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.403049 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87f2b-6853-4e52-b0e3-153e85bb1814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.431884 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-c4d87f2b-6853-4e52-b0e3-153e85bb1814 None None] MFA Rules not processed for user `d45fd67692aa4a3e88c0929b0a7de9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:45.433103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c4d87f2b-6853-4e52-b0e3-153e85bb1814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:45.496620 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1447/2881] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:45.501444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.559856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070d92c-e404-4a05-add0-08b42e31b7c0 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.564344 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.564925 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.565198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.565496 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.566060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.567954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.632477 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1435/2882] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:20:45 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:45.636970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d87f2b-6853-4e52-b0e3-153e85bb1814 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.647683 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d87f2b-6853-4e52-b0e3-153e85bb1814 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.702515 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.703137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.703425 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.703690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.704322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.706203 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.713877 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:45.762953 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1448/2883] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:20:45 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:45.766954 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.777301 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e346c-9475-4320-8af3-fa81018bebf4 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.782368 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.783059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.783365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.783657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.784337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.785965 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.797022 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1436/2884] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:45.810273 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.820163 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cc4a7-39d2-4115-8216-7720c129d14f admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.825381 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.826012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.826284 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.826552 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] PATH_INFO: `/v3/projects/5255bccfb8bd45598ac8446348e615c5/users/24c87b56b4d841e6a019ad8a27d046e3/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.841016 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5255bccfb8bd45598ac8446348e615c5, user_id=24c87b56b4d841e6a019ad8a27d046e3, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.843649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.922698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1449/2885] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:45 2022] PUT /identity/v3/projects/5255bccfb8bd45598ac8446348e615c5/users/24c87b56b4d841e6a019ad8a27d046e3/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:45.926525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:45.937170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524180-42fd-4c64-bd49-5376093f2345 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:45.991454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c70e13-f0e6-4c84-95e4-05dc6ea47932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:45.992048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70e13-f0e6-4c84-95e4-05dc6ea47932 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:45.992325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70e13-f0e6-4c84-95e4-05dc6ea47932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:45.993105 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70e13-f0e6-4c84-95e4-05dc6ea47932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:45.993729 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c70e13-f0e6-4c84-95e4-05dc6ea47932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:45.995334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c70e13-f0e6-4c84-95e4-05dc6ea47932 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:45.997319 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:46.005563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1437/2886] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:20:46.010277 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:46.020592 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c25d4-9b85-4e06-9afd-23ec1fcb5f5a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:46.025081 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:46.033274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:46.033274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:46.033274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] PATH_INFO: `/v3/projects/5255bccfb8bd45598ac8446348e615c5/users/24c87b56b4d841e6a019ad8a27d046e3/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:46.033274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5255bccfb8bd45598ac8446348e615c5, user_id=24c87b56b4d841e6a019ad8a27d046e3, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:46.033274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:46.041531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1450/2887] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:20:46 2022] PUT /identity/v3/projects/5255bccfb8bd45598ac8446348e615c5/users/24c87b56b4d841e6a019ad8a27d046e3/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:46.052646 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a90dd-6540-4c97-b0b2-f5277be81378 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:46.052938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a90dd-6540-4c97-b0b2-f5277be81378 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:46.053229 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a90dd-6540-4c97-b0b2-f5277be81378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:46.080961 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-650a90dd-6540-4c97-b0b2-f5277be81378 None None] MFA Rules not processed for user `24c87b56b4d841e6a019ad8a27d046e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:46.082147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-650a90dd-6540-4c97-b0b2-f5277be81378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:46.147690 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1438/2888] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:20:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:20:46.163773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:46.166152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:46.222264 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24991c-fd9f-4d6b-a510-a11f05e5077a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:46.225944 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:46.226534 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:46.227358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:46.227629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:46.236574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:46.237995 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:46.248184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1451/2889] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:46.485065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a90dd-6540-4c97-b0b2-f5277be81378 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:46.495879 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a90dd-6540-4c97-b0b2-f5277be81378 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:46.499782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee482508-24fd-457f-a73f-4eb0afde44d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:46.500365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee482508-24fd-457f-a73f-4eb0afde44d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:46.500638 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee482508-24fd-457f-a73f-4eb0afde44d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:46.500904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee482508-24fd-457f-a73f-4eb0afde44d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:46.556005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee482508-24fd-457f-a73f-4eb0afde44d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:46.557436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee482508-24fd-457f-a73f-4eb0afde44d8 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:46.566131 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1439/2890] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:20:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:20:56.243869 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:INSERT=5,keystone:SELECT=43 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:56.562283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=63 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:00.271423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:00.332311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b8524-67b0-45aa-b67d-52032ea523e6 service cinder] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:00.337068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:00.337681 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:00.337961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:00.338262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] PATH_INFO: `/v3/users/d45fd67692aa4a3e88c0929b0a7de9ab` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:00.339438 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d45fd67692aa4a3e88c0929b0a7de9ab)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:00.341337 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:00.537029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd45fd67692aa4a3e88c0929b0a7de9ab'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:00.537616 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd45fd67692aa4a3e88c0929b0a7de9ab'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:00.591342 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd45fd67692aa4a3e88c0929b0a7de9ab'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:00.604174 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd45fd67692aa4a3e88c0929b0a7de9ab'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:00.604821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd45fd67692aa4a3e88c0929b0a7de9ab'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:00.616641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1452/2891] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:21:00 2022] DELETE /identity/v3/users/d45fd67692aa4a3e88c0929b0a7de9ab => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:00.621293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:00.621771 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:00.621867 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:00.641818 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:00.643041 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:00.699765 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1440/2892] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:00.709565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:00.711010 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:00.767810 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d7ec6-ada1-409b-9faa-883da49788ca admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:00.771496 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:00.772089 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:00.772368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:00.772643 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:00.781466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:00.782916 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:00.792269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1453/2893] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:21:01.192993 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:01.194529 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:01.206334 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3d732-9142-46a0-ab28-12f51a4cf34f None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:01.211346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:01.212152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:01.212434 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:01.212892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] PATH_INFO: `/v3/projects/ef22d0c9df444e05a90135ea0c91bf3e` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:01.213892 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef22d0c9df444e05a90135ea0c91bf3e)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:01.216746 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:01.351137 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef22d0c9df444e05a90135ea0c91bf3e', 'request_id': 'req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.357661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef22d0c9df444e05a90135ea0c91bf3e', 'request_id': 'req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.363081 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef22d0c9df444e05a90135ea0c91bf3e', 'request_id': 'req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.431744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] The token cache is being invalidate because project ef22d0c9df444e05a90135ea0c91bf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:21:01.432037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.432655 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.434158 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1441/2894] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:21:01 2022] DELETE /identity/v3/projects/ef22d0c9df444e05a90135ea0c91bf3e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:01.438271 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:01.447993 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360a0b9-73be-4ad9-bc76-f8a64513c3e1 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:01.466298 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:01.504574 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:01.505378 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:01.505829 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:01.506533 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] PATH_INFO: `/v3/users/24c87b56b4d841e6a019ad8a27d046e3` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:01.507661 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c87b56b4d841e6a019ad8a27d046e3)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:01.509563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:01.701698 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c87b56b4d841e6a019ad8a27d046e3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.702299 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c87b56b4d841e6a019ad8a27d046e3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.763069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c87b56b4d841e6a019ad8a27d046e3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.775917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c87b56b4d841e6a019ad8a27d046e3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.776543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-394b8a44-ee56-449f-b3a0-1fc2b2906f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c87b56b4d841e6a019ad8a27d046e3'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:01.789512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1454/2895] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:21:01 2022] DELETE /identity/v3/users/24c87b56b4d841e6a019ad8a27d046e3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:02.164373 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:02.165819 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:02.179184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be77ab7-b1fe-46fb-bc19-6b5c1cecdbdb admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:02.234616 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:02.235258 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.235530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.235793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] PATH_INFO: `/v3/projects/5255bccfb8bd45598ac8446348e615c5` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.236933 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5255bccfb8bd45598ac8446348e615c5)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:02.238782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:02.328632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:02.343973 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5255bccfb8bd45598ac8446348e615c5', 'request_id': 'req-b1fbb453-72b2-4868-b5cb-1e52fcd18491'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:02.349782 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5255bccfb8bd45598ac8446348e615c5', 'request_id': 'req-b1fbb453-72b2-4868-b5cb-1e52fcd18491'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:02.355098 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5255bccfb8bd45598ac8446348e615c5', 'request_id': 'req-b1fbb453-72b2-4868-b5cb-1e52fcd18491'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:02.447687 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] The token cache is being invalidate because project 5255bccfb8bd45598ac8446348e615c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:21:02.447980 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:02.448610 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b1fbb453-72b2-4868-b5cb-1e52fcd18491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:02.450130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1442/2896] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:21:02 2022] DELETE /identity/v3/projects/5255bccfb8bd45598ac8446348e615c5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:02.455183 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb13930-d789-44c6-8883-fdb3c273660a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.455666 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb13930-d789-44c6-8883-fdb3c273660a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.455770 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb13930-d789-44c6-8883-fdb3c273660a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.475241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-bcb13930-d789-44c6-8883-fdb3c273660a None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:02.476485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-bcb13930-d789-44c6-8883-fdb3c273660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:02.536113 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1455/2897] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:02.543847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e060530-abe8-4c5e-9f06-db132d51b7ac None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.544182 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e060530-abe8-4c5e-9f06-db132d51b7ac None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.544457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e060530-abe8-4c5e-9f06-db132d51b7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.563906 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-8e060530-abe8-4c5e-9f06-db132d51b7ac None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:02.565101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8e060530-abe8-4c5e-9f06-db132d51b7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:02.573480 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1443/2898] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:02.577509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb13930-d789-44c6-8883-fdb3c273660a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:02.587380 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb13930-d789-44c6-8883-fdb3c273660a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:02.591934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:02.592517 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.592795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.593069 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] PATH_INFO: `/v3/domains` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.593704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81680) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:21:02.594003 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:02.595755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:02.607636 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1456/2899] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:21:02.611657 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e060530-abe8-4c5e-9f06-db132d51b7ac None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:02.621376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e060530-abe8-4c5e-9f06-db132d51b7ac None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:02.626222 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:02.626840 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.627121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.627389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.628016 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:02.629913 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:02.747975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1444/2900] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:21:02 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:21:02.752207 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:02.762186 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb61efc-6864-4ab6-a7cc-557fd860c05c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:02.787111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:02.817156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:02.817768 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.818033 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.818513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.819362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:02.821727 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:02.934519 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1457/2901] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:21:02 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:21:02.938821 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:02.949637 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e7c7d-aaed-404d-bd93-05a9c4db243b admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:02.954397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:02.955046 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.955325 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.955592 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.956220 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:02.957790 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:02.967461 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1445/2902] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:02.971232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:02.980775 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca567fb-04fc-44b1-8e85-a23b66c2d025 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:02.985251 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b2c11-df2c-4a3f-a5b8-c47a8816be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:02.985835 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b2c11-df2c-4a3f-a5b8-c47a8816be7b admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:02.986110 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b2c11-df2c-4a3f-a5b8-c47a8816be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:02.986379 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b2c11-df2c-4a3f-a5b8-c47a8816be7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:02.987047 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b2c11-df2c-4a3f-a5b8-c47a8816be7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:02.988605 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b2c11-df2c-4a3f-a5b8-c47a8816be7b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:02.998891 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1458/2903] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:03.002676 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.012724 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026ff07-391b-454f-aafc-ae1f65c23ae4 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:03.017107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:03.017726 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.018001 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.018271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] PATH_INFO: `/v3/projects/edad12ab25b94884954c28cb89034a51/users/d55da0e361e14aaa953035573b2fb088/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.032432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=edad12ab25b94884954c28cb89034a51, user_id=d55da0e361e14aaa953035573b2fb088, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:03.035057 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:03.128143 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1446/2904] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:21:03 2022] PUT /identity/v3/projects/edad12ab25b94884954c28cb89034a51/users/d55da0e361e14aaa953035573b2fb088/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:03.140285 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd8fd1-d95e-44ea-82e9-db2e7e81654e None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.141332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd8fd1-d95e-44ea-82e9-db2e7e81654e None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.141332 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd8fd1-d95e-44ea-82e9-db2e7e81654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.171241 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ffdd8fd1-d95e-44ea-82e9-db2e7e81654e None None] MFA Rules not processed for user `d55da0e361e14aaa953035573b2fb088`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:03.172491 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ffdd8fd1-d95e-44ea-82e9-db2e7e81654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:03.240556 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1459/2905] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:03.245612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.308441 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05244ece-0a6a-4c7a-a819-92204a29d52c admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:03.313028 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:03.313620 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.313897 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.314167 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] PATH_INFO: `/v3/projects` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.314804 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:03.316682 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:03.432357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1447/2906] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:21:03 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:21:03.436555 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd8fd1-d95e-44ea-82e9-db2e7e81654e None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.447843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd8fd1-d95e-44ea-82e9-db2e7e81654e None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:03.502269 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:03.503630 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.504071 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.504584 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] PATH_INFO: `/v3/users` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.505442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:03.508075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:03.586957 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1460/2907] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:21:03 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:21:03.591320 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.601807 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9885c-f22e-42b2-9917-0370a459167d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:03.606454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:03.607070 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.607353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.607627 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.608263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:03.609847 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:03.620130 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1448/2908] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:03.624451 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.634983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd073a3-2ed9-4a6a-b134-b7286a8dd91c admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:03.639570 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:03.640220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.640512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.640787 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] PATH_INFO: `/v3/projects/87e8944d1b7b4574ab23038e5cb4c25e/users/4cc3051d4d2d4e01821b54fb96705ce5/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.656187 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e8944d1b7b4574ab23038e5cb4c25e, user_id=4cc3051d4d2d4e01821b54fb96705ce5, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:03.658860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:03.716960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1461/2909] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:21:03 2022] PUT /identity/v3/projects/87e8944d1b7b4574ab23038e5cb4c25e/users/4cc3051d4d2d4e01821b54fb96705ce5/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:03.720869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.731191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a3d30-2fbb-4661-82f3-cf722edf4e6a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:03.751476 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:03.786931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386ffcf-8f4e-4f41-9792-7cd859a2080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:03.787528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ffcf-8f4e-4f41-9792-7cd859a2080d admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.787809 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ffcf-8f4e-4f41-9792-7cd859a2080d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.788084 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ffcf-8f4e-4f41-9792-7cd859a2080d admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.788734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386ffcf-8f4e-4f41-9792-7cd859a2080d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:03.790304 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386ffcf-8f4e-4f41-9792-7cd859a2080d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:03.799801 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1449/2910] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:03.803921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.809859 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:03.816158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49f99f-7c50-4822-93e4-079b1a9de97e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:03.821424 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:03.822092 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.822370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.822667 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] PATH_INFO: `/v3/projects/87e8944d1b7b4574ab23038e5cb4c25e/users/4cc3051d4d2d4e01821b54fb96705ce5/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.828444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e8944d1b7b4574ab23038e5cb4c25e, user_id=4cc3051d4d2d4e01821b54fb96705ce5, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:03.828444 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:03.837338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1462/2911] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:21:03 2022] PUT /identity/v3/projects/87e8944d1b7b4574ab23038e5cb4c25e/users/4cc3051d4d2d4e01821b54fb96705ce5/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:03.847450 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acaed9-ee4a-40c1-852d-52f4a995d82b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:03.847737 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acaed9-ee4a-40c1-852d-52f4a995d82b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:03.848011 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acaed9-ee4a-40c1-852d-52f4a995d82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:03.877179 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-a0acaed9-ee4a-40c1-852d-52f4a995d82b None None] MFA Rules not processed for user `4cc3051d4d2d4e01821b54fb96705ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:03.878377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-a0acaed9-ee4a-40c1-852d-52f4a995d82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:03.944975 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1450/2912] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:03.956955 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:03.958691 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:04.015295 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a41e1-ccca-401a-97b7-a70d60eb9542 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:04.019104 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0511d-4047-43cb-b1c2-fe26d5b5d016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:04.019704 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0511d-4047-43cb-b1c2-fe26d5b5d016 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:04.019983 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0511d-4047-43cb-b1c2-fe26d5b5d016 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:04.020259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0511d-4047-43cb-b1c2-fe26d5b5d016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:04.029405 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0511d-4047-43cb-b1c2-fe26d5b5d016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:04.030866 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0511d-4047-43cb-b1c2-fe26d5b5d016 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:04.040139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1463/2913] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:21:05.091432 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acaed9-ee4a-40c1-852d-52f4a995d82b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:05.151701 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acaed9-ee4a-40c1-852d-52f4a995d82b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:05.156078 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:05.156688 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:05.156960 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:05.157227 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] PATH_INFO: `/v3/users/d55da0e361e14aaa953035573b2fb088` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:05.158326 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55da0e361e14aaa953035573b2fb088)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:05.160226 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:05.312917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55da0e361e14aaa953035573b2fb088'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:05.313566 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55da0e361e14aaa953035573b2fb088'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:05.314184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55da0e361e14aaa953035573b2fb088'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:05.315383 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:05.321548 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55da0e361e14aaa953035573b2fb088'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:05.332802 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55da0e361e14aaa953035573b2fb088'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:05.436955 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1451/2914] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:21:05 2022] DELETE /identity/v3/users/d55da0e361e14aaa953035573b2fb088 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:05.441445 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:05.441912 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:05.442009 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:05.460420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:05.462129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:05.519149 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1464/2915] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:05.529337 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:05.530952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:05.587166 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b0aa3-a8cf-45ea-a805-e336e80b763d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:05.590700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:05.591296 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:05.591578 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:05.591850 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:05.600612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:05.602103 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:05.611348 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1452/2916] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:21:06.026795 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:06.028232 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:06.042349 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c241ba-e299-4881-b823-e345c874abe0 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:06.047046 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:06.047665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:06.048437 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:06.048710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] PATH_INFO: `/v3/projects/edad12ab25b94884954c28cb89034a51` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:06.050001 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edad12ab25b94884954c28cb89034a51)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:06.051971 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:06.162858 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edad12ab25b94884954c28cb89034a51', 'request_id': 'req-24f1712a-c503-413e-8123-c31c3a9f8b89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.168289 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edad12ab25b94884954c28cb89034a51', 'request_id': 'req-24f1712a-c503-413e-8123-c31c3a9f8b89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.175154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edad12ab25b94884954c28cb89034a51', 'request_id': 'req-24f1712a-c503-413e-8123-c31c3a9f8b89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.270409 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] The token cache is being invalidate because project edad12ab25b94884954c28cb89034a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:21:06.270718 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.271360 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.272847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1465/2917] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:21:06 2022] DELETE /identity/v3/projects/edad12ab25b94884954c28cb89034a51 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:06.276970 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:06.286347 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47f138-acb8-44f1-8fe4-abcb8de6fd60 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:06.341613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:06.342152 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:06.342427 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:06.342721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] PATH_INFO: `/v3/users/4cc3051d4d2d4e01821b54fb96705ce5` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:06.343816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc3051d4d2d4e01821b54fb96705ce5)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:06.345702 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:06.559281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc3051d4d2d4e01821b54fb96705ce5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.559977 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc3051d4d2d4e01821b54fb96705ce5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.560584 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc3051d4d2d4e01821b54fb96705ce5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.567805 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc3051d4d2d4e01821b54fb96705ce5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.578079 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:06.580051 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-b827596e-fe2a-4c19-adc4-10a9731ad9fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc3051d4d2d4e01821b54fb96705ce5'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:06.659395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1453/2918] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:21:06 2022] DELETE /identity/v3/users/4cc3051d4d2d4e01821b54fb96705ce5 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:07.049083 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:07.050558 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:07.064126 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1712a-c503-413e-8123-c31c3a9f8b89 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:07.085314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:07.131365 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:07.131985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.132260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.132525 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] PATH_INFO: `/v3/projects/87e8944d1b7b4574ab23038e5cb4c25e` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.134178 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e8944d1b7b4574ab23038e5cb4c25e)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:07.136124 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:07.248028 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e8944d1b7b4574ab23038e5cb4c25e', 'request_id': 'req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:07.253550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e8944d1b7b4574ab23038e5cb4c25e', 'request_id': 'req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:07.260252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e8944d1b7b4574ab23038e5cb4c25e', 'request_id': 'req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:07.343914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] The token cache is being invalidate because project 87e8944d1b7b4574ab23038e5cb4c25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:21:07.344202 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:07.344828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-3fd329d4-6449-4b18-b6c0-c7ffefebdf47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:07.346311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1466/2919] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:21:07 2022] DELETE /identity/v3/projects/87e8944d1b7b4574ab23038e5cb4c25e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:07.351365 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5323edd-c956-470c-ae28-2eb660e27275 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.351932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5323edd-c956-470c-ae28-2eb660e27275 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.351932 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5323edd-c956-470c-ae28-2eb660e27275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.370693 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-b5323edd-c956-470c-ae28-2eb660e27275 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:07.371903 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b5323edd-c956-470c-ae28-2eb660e27275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:07.432917 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1454/2920] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:07.440773 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80950-cb3a-4e8d-a69f-1151ef39efec None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.441109 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80950-cb3a-4e8d-a69f-1151ef39efec None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.444173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80950-cb3a-4e8d-a69f-1151ef39efec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.462079 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-94a80950-cb3a-4e8d-a69f-1151ef39efec None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:07.463402 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-94a80950-cb3a-4e8d-a69f-1151ef39efec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:07.471418 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1467/2921] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:21:07.475530 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5323edd-c956-470c-ae28-2eb660e27275 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:07.485086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5323edd-c956-470c-ae28-2eb660e27275 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:07.489764 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:07.490346 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.490618 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.490907 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.491536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:21:07.491828 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:07.493938 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:07.505456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1455/2922] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:21:07.510011 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80950-cb3a-4e8d-a69f-1151ef39efec None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:07.518940 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80950-cb3a-4e8d-a69f-1151ef39efec None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:07.523567 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:07.524139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.524413 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.524679 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.525322 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:07.527262 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:07.621327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1468/2923] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:21:07 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:21:07.625669 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:07.636424 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84152dba-6d0b-4c12-8fae-2daf32bec774 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:07.690860 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:07.691456 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.691734 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.692005 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.692632 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:07.694536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:07.802707 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1456/2924] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:21:07 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:21:07.807561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:07.818246 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b483ce-35c3-4425-aa15-96a28fa670ea admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:07.823583 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:07.824184 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.824462 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.824735 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.825370 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:07.827076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:07.836827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1469/2925] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:21:07.840557 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:07.850331 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7091e49-5937-46c4-9ac2-8b67604f7e6d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:07.855658 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d823b9a-fec1-4375-9d0c-1e0a98948a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:07.856248 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d823b9a-fec1-4375-9d0c-1e0a98948a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.856570 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d823b9a-fec1-4375-9d0c-1e0a98948a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.856841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d823b9a-fec1-4375-9d0c-1e0a98948a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.857501 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d823b9a-fec1-4375-9d0c-1e0a98948a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:07.859153 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d823b9a-fec1-4375-9d0c-1e0a98948a01 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:07.870302 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1457/2926] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:07.874327 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:07.884333 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b1037-1a69-4e7d-87d0-abdb0daeae6e admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:07.889156 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:07.889794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:07.890074 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:07.890347 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] PATH_INFO: `/v3/projects/9945c837d42a493db6b274d465a846a5/users/f2bc87d4e2a74681ac1c6f6aab6a5150/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:07.904651 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9945c837d42a493db6b274d465a846a5, user_id=f2bc87d4e2a74681ac1c6f6aab6a5150, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:07.907306 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:07.997423 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1470/2927] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:21:07 2022] PUT /identity/v3/projects/9945c837d42a493db6b274d465a846a5/users/f2bc87d4e2a74681ac1c6f6aab6a5150/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:08.069261 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574eb4a-7103-43a5-9b7a-cf7fdee59c5b None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:08.069449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574eb4a-7103-43a5-9b7a-cf7fdee59c5b None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:08.069715 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574eb4a-7103-43a5-9b7a-cf7fdee59c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:08.096200 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-e574eb4a-7103-43a5-9b7a-cf7fdee59c5b None None] MFA Rules not processed for user `f2bc87d4e2a74681ac1c6f6aab6a5150`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:08.097436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e574eb4a-7103-43a5-9b7a-cf7fdee59c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:08.137630 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:08.159361 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1458/2928] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:08.170879 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:08.172559 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:08.218229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:08.230188 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc438-92d5-491c-9afe-6680fdaf4a58 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:08.233959 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c8359-052c-42ee-96e1-7a5e2e423c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:08.234547 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8359-052c-42ee-96e1-7a5e2e423c94 service cinder] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:08.234853 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8359-052c-42ee-96e1-7a5e2e423c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:08.235138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8359-052c-42ee-96e1-7a5e2e423c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:08.243486 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c8359-052c-42ee-96e1-7a5e2e423c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:08.244892 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c8359-052c-42ee-96e1-7a5e2e423c94 service cinder] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:08.253708 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1471/2929] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:21:08.280585 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574eb4a-7103-43a5-9b7a-cf7fdee59c5b None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:08.343416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574eb4a-7103-43a5-9b7a-cf7fdee59c5b None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:08.347798 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:08.348408 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:08.348677 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:08.348941 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] PATH_INFO: `/v3/users/f2bc87d4e2a74681ac1c6f6aab6a5150` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:08.350086 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2bc87d4e2a74681ac1c6f6aab6a5150)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:08.352606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:08.490506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2bc87d4e2a74681ac1c6f6aab6a5150'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:08.491189 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2bc87d4e2a74681ac1c6f6aab6a5150'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:08.491795 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2bc87d4e2a74681ac1c6f6aab6a5150'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:08.499032 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2bc87d4e2a74681ac1c6f6aab6a5150'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:08.515126 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2bc87d4e2a74681ac1c6f6aab6a5150'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:08.567665 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1459/2930] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:21:08 2022] DELETE /identity/v3/users/f2bc87d4e2a74681ac1c6f6aab6a5150 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:08.572721 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:08.573921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:08.573921 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:08.592399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:08.593641 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:08.650550 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1472/2931] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:08.660184 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:08.661712 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:08.716859 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5077-06f7-48c6-a31b-7714c73aa21d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:08.720516 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad01799-c863-40fa-a9d4-1cd57f35941c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:08.721110 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01799-c863-40fa-a9d4-1cd57f35941c service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:08.721389 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01799-c863-40fa-a9d4-1cd57f35941c service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:08.721686 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad01799-c863-40fa-a9d4-1cd57f35941c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:08.730750 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad01799-c863-40fa-a9d4-1cd57f35941c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:08.732177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad01799-c863-40fa-a9d4-1cd57f35941c service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:08.741507 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1460/2932] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:21:09.142073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:09.143649 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:09.158758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d302c0-19da-4b37-bcbe-16a6587f980d None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:09.163634 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:09.164411 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.164755 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.164946 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] PATH_INFO: `/v3/projects/9945c837d42a493db6b274d465a846a5` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.166137 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9945c837d42a493db6b274d465a846a5)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:09.168067 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:09.277754 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9945c837d42a493db6b274d465a846a5', 'request_id': 'req-f52d347f-f773-4328-8e15-33f3258a6091'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:09.283715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9945c837d42a493db6b274d465a846a5', 'request_id': 'req-f52d347f-f773-4328-8e15-33f3258a6091'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:09.292696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9945c837d42a493db6b274d465a846a5', 'request_id': 'req-f52d347f-f773-4328-8e15-33f3258a6091'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:09.362103 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] The token cache is being invalidate because project 9945c837d42a493db6b274d465a846a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81680) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:21:09.362392 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:09.363063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-f52d347f-f773-4328-8e15-33f3258a6091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:21:09.364601 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1473/2933] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:21:09 2022] DELETE /identity/v3/projects/9945c837d42a493db6b274d465a846a5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:09.370138 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423bda9-9240-4b07-a0d5-bf2bfa0700d7 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.370622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423bda9-9240-4b07-a0d5-bf2bfa0700d7 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.370748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423bda9-9240-4b07-a0d5-bf2bfa0700d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.379493 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:09.390010 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0423bda9-9240-4b07-a0d5-bf2bfa0700d7 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:09.391253 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0423bda9-9240-4b07-a0d5-bf2bfa0700d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:09.449407 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1461/2934] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:09.456905 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac7d26-ff3d-4326-9ae8-7fa950799849 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.458830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac7d26-ff3d-4326-9ae8-7fa950799849 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.458830 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac7d26-ff3d-4326-9ae8-7fa950799849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.477235 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-7cac7d26-ff3d-4326-9ae8-7fa950799849 None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:09.478448 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-7cac7d26-ff3d-4326-9ae8-7fa950799849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:09.486244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1474/2935] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:09.490293 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423bda9-9240-4b07-a0d5-bf2bfa0700d7 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:09.500880 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423bda9-9240-4b07-a0d5-bf2bfa0700d7 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:09.505449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:09.506146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.506428 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.506797 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] PATH_INFO: `/v3/domains` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.507444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81681) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:21:09.507744 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:09.509354 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:09.520685 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1462/2936] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 13:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:21:09.524828 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac7d26-ff3d-4326-9ae8-7fa950799849 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:09.533926 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac7d26-ff3d-4326-9ae8-7fa950799849 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:09.538553 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:09.539716 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.539991 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.540258 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.540878 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:09.542818 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:09.632899 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1475/2937] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:21:09 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:21:09.637149 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:09.647101 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461519b1-f7b5-49af-a5b0-d8cddd63b03a admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:09.702716 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:09.703240 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.703519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.703837 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.704454 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:09.706345 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:09.784489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1463/2938] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:21:09 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:21:09.788563 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:09.799101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ee7d2-6f95-4d31-81be-96949da0ff45 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:09.803714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:09.804303 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.805098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.805376 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.806012 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:09.807633 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:09.817363 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1476/2939] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:21:09.821793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:09.831551 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7355f43-26ef-492a-a70f-9da3a3757b6d admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:09.836014 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ab544-bbc6-43cc-84ac-d01379864d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:09.836591 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab544-bbc6-43cc-84ac-d01379864d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.836868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab544-bbc6-43cc-84ac-d01379864d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.837139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab544-bbc6-43cc-84ac-d01379864d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.837770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ab544-bbc6-43cc-84ac-d01379864d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:09.839393 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ab544-bbc6-43cc-84ac-d01379864d60 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:09.849587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1464/2940] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:09.853758 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:09.864450 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041699b0-1936-4c99-9e42-db1865daf311 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:09.869460 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:09.870088 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:09.870362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:09.870671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] PATH_INFO: `/v3/projects/e4ab80b71d1f4f06b20c597689ae8a2e/users/cc07ef53f88c4a809deab768daff8132/roles/dbcd6da9a237441cb483965b19c21cff` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:09.885587 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ab80b71d1f4f06b20c597689ae8a2e, user_id=cc07ef53f88c4a809deab768daff8132, role_id=dbcd6da9a237441cb483965b19c21cff)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:09.888229 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:09.996543 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1477/2941] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:21:09 2022] PUT /identity/v3/projects/e4ab80b71d1f4f06b20c597689ae8a2e/users/cc07ef53f88c4a809deab768daff8132/roles/dbcd6da9a237441cb483965b19c21cff => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:10.001770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4364a0-c485-46c8-af05-2698cf09a55e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:10.002060 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4364a0-c485-46c8-af05-2698cf09a55e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:10.002336 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4364a0-c485-46c8-af05-2698cf09a55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:10.021436 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-5c4364a0-c485-46c8-af05-2698cf09a55e None None] MFA Rules not processed for user `ad9be8700f45469d8fb5f44fbb2eeb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:10.022667 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-5c4364a0-c485-46c8-af05-2698cf09a55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:10.079042 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1465/2942] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:10.088860 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:10.090575 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:10.097008 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:10.148373 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381515a-5ded-456d-8362-1de0fd9c37ca admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:10.152243 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:10.152821 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:10.153210 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:10.153485 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:10.163002 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:10.164934 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:10.174274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1478/2943] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:21:15.508135 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60983801-a6f8-44d4-94f2-2ea45a2ce46e None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:15.508661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60983801-a6f8-44d4-94f2-2ea45a2ce46e None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:15.508661 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60983801-a6f8-44d4-94f2-2ea45a2ce46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:15.532586 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:15.541541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-60983801-a6f8-44d4-94f2-2ea45a2ce46e None None] MFA Rules not processed for user `cc07ef53f88c4a809deab768daff8132`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:15.545740 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-60983801-a6f8-44d4-94f2-2ea45a2ce46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:15.613470 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1466/2944] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:15.617990 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:15.628498 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec29091-b030-4bed-9e41-02fa52e18396 service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:15.634158 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:15.634813 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:15.635093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:15.635362 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:15.636029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:15.637979 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:15.733314 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1479/2945] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 13:21:15 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:21:15.734587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60983801-a6f8-44d4-94f2-2ea45a2ce46e None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:15.746492 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60983801-a6f8-44d4-94f2-2ea45a2ce46e None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:15.805723 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:15.809035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:15.809035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:15.809035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] PATH_INFO: `/v3/users` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:15.809035 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:15.809468 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:15.899651 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1467/2946] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 13:21:15 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:21:15.903590 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:15.918195 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a264a02-0fa6-4e27-973e-f5b0f722d7bf admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:15.923352 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:15.925582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:15.925582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:15.925582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:15.925582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:15.926840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:15.944244 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1480/2947] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:15.946946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:15.962457 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337780c-b896-4f6a-9512-2061a2d3dd59 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:15.967758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:15.968440 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:15.968721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:15.968998 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] PATH_INFO: `/v3/projects/7461b8d063104ad0b8da1e8bdf9ee73f/users/8df5b9f369bf491b9ef61bc45ad5cd89/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:15.985936 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7461b8d063104ad0b8da1e8bdf9ee73f, user_id=8df5b9f369bf491b9ef61bc45ad5cd89, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:15.989948 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:16.065611 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1468/2948] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:21:15 2022] PUT /identity/v3/projects/7461b8d063104ad0b8da1e8bdf9ee73f/users/8df5b9f369bf491b9ef61bc45ad5cd89/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:16.069731 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:16.080319 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff859f-31e5-4af3-b814-c9acc6bea86a admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:16.138764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856af6b-8415-4a4b-9e85-d3ed6ea23283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:16.139386 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856af6b-8415-4a4b-9e85-d3ed6ea23283 admin admin] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:16.139669 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856af6b-8415-4a4b-9e85-d3ed6ea23283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:16.139950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856af6b-8415-4a4b-9e85-d3ed6ea23283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:16.140589 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856af6b-8415-4a4b-9e85-d3ed6ea23283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:16.142196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856af6b-8415-4a4b-9e85-d3ed6ea23283 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:16.152466 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1481/2949] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 13:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:21:16.156751 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:16.167590 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b15f2-b019-451b-abe8-a8ae26c4d0ff admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:16.172191 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:16.172773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:16.173053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:16.173327 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] PATH_INFO: `/v3/projects/7461b8d063104ad0b8da1e8bdf9ee73f/users/8df5b9f369bf491b9ef61bc45ad5cd89/roles/753fd078fdac40abb65aa43057d79c62` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:16.175269 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7461b8d063104ad0b8da1e8bdf9ee73f, user_id=8df5b9f369bf491b9ef61bc45ad5cd89, role_id=753fd078fdac40abb65aa43057d79c62)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:16.177869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:16.188504 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1469/2950] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 13:21:16 2022] PUT /identity/v3/projects/7461b8d063104ad0b8da1e8bdf9ee73f/users/8df5b9f369bf491b9ef61bc45ad5cd89/roles/753fd078fdac40abb65aa43057d79c62 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:21:21.115152 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df7de-e0fb-4067-b0d2-ece6e2797589 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:21.115876 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df7de-e0fb-4067-b0d2-ece6e2797589 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:21.115980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760df7de-e0fb-4067-b0d2-ece6e2797589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:21.145467 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-760df7de-e0fb-4067-b0d2-ece6e2797589 None None] MFA Rules not processed for user `8df5b9f369bf491b9ef61bc45ad5cd89`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:21.146779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-760df7de-e0fb-4067-b0d2-ece6e2797589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:21.209705 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:21.219531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1482/2951] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:21.236088 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:21.238147 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:21.306053 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:21.329864 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2270092-7514-47d7-9cdc-9da25be21c96 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:21.338198 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f983c67-e4f2-45fc-911a-e640d7e82b3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:21.338916 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f983c67-e4f2-45fc-911a-e640d7e82b3d service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:21.339257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f983c67-e4f2-45fc-911a-e640d7e82b3d service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:21.339587 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f983c67-e4f2-45fc-911a-e640d7e82b3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:21.404607 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f983c67-e4f2-45fc-911a-e640d7e82b3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:21.406353 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f983c67-e4f2-45fc-911a-e640d7e82b3d service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:21.416489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1470/2952] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:21:31.374307 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:31.487604 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=32 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:21:48.794856 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:48.795335 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:48.795521 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:48.814101 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] MFA Rules not processed for user `cc07ef53f88c4a809deab768daff8132`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:48.815311 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:48.825822 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1483/2953] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:21:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:50.901257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:50.901257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:50.901257 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:50.938146 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:21:50.939911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:51.001700 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1471/2954] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 13:21:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:21:51.013783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:51.015214 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:51.030305 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79129da1-6ec8-4ce9-9b22-f3663c5caba6 None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:51.035123 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:51.035710 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:51.035985 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:51.036254 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:51.045030 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:51.046573 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:51.055950 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1484/2955] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:21:54.462959 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:21:54.464487 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:54.532705 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755a07d-01b5-4499-9381-746b89ba5811 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:21:54.536303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:21:54.536902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:21:54.537176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:21:54.537445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:21:54.539779 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:21:54.549255 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:21:54.550642 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:21:54.559391 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1472/2956] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:21:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:22:01.052068 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=11 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:04.556282 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=65 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:10.835209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:10.848018 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240e510-ee9b-4dda-bd4c-f641b18e2bfc service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:10.853098 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:10.853843 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:10.854118 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:10.854383 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:10.863786 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:10.865198 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:10.875399 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1485/2957] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:22:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:22:20.870338 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:24:28.311300 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:24:28.320175 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8bede-4af9-43b1-88bd-a4a830d371fb service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:24:28.323760 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:24:28.324519 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:28.324761 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:28.325037 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:28.326773 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:28.339832 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:24:28.341236 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:24:28.352506 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1473/2958] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:24:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:24:29.533520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:24:29.541933 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743872c-c7f4-49cb-baa8-9d712d01ad6c service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:24:29.546005 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2f8f4-9467-474f-b1fa-7667ee0b7a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:24:29.546578 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2f8f4-9467-474f-b1fa-7667ee0b7a7f service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:29.546882 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2f8f4-9467-474f-b1fa-7667ee0b7a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:29.547151 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2f8f4-9467-474f-b1fa-7667ee0b7a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:29.548844 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-a8f2f8f4-9467-474f-b1fa-7667ee0b7a7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:29.562426 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2f8f4-9467-474f-b1fa-7667ee0b7a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:24:29.564580 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2f8f4-9467-474f-b1fa-7667ee0b7a7f service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:24:29.575847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1486/2959] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:24:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:24:31.273377 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:24:31.275065 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:31.289209 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e46a1-0daa-4749-8ca3-825a5b9fb9a4 service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:24:31.294793 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:24:31.295499 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:31.296289 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:31.296559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:31.298281 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:31.310426 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:24:31.318285 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:24:31.335843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1474/2960] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:24:33.053025 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1c8c-a717-46dc-bf56-6d8ae245fb83 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:33.053513 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1c8c-a717-46dc-bf56-6d8ae245fb83 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:33.053611 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1c8c-a717-46dc-bf56-6d8ae245fb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:33.071943 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-e90c1c8c-a717-46dc-bf56-6d8ae245fb83 None None] MFA Rules not processed for user `cc07ef53f88c4a809deab768daff8132`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:24:33.073139 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-e90c1c8c-a717-46dc-bf56-6d8ae245fb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:33.080520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1487/2961] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:24:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:24:33.391606 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:24:33.400812 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215df2e-9f0c-4f10-8d2f-fc768e347e1f service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:24:33.404416 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:24:33.404999 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:33.405275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:33.405543 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:33.407275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:33.416706 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:24:33.418121 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:24:33.426691 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1475/2962] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:24:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:24:34.135275 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe8495-bcb3-4e60-9593-b647092abc5a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:34.135736 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe8495-bcb3-4e60-9593-b647092abc5a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:34.135842 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe8495-bcb3-4e60-9593-b647092abc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:34.155746 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-ebfe8495-bcb3-4e60-9593-b647092abc5a None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:24:34.156973 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-ebfe8495-bcb3-4e60-9593-b647092abc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:34.165696 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1488/2963] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 13:24:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:24:34.178017 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:24:34.189931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e9797-b14c-4ca1-8122-586bd574680d service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:24:34.194770 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:24:34.195357 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:34.195634 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:34.195904 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:34.203986 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:24:34.205540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:24:34.215059 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1476/2964] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:24:44.149715 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=12 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:24:44.210994 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=24 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:24:50.508504 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe8495-bcb3-4e60-9593-b647092abc5a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:24:50.520794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe8495-bcb3-4e60-9593-b647092abc5a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:24:50.526164 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d66a7-3904-41f6-8151-6051852bf1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:24:50.530122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d66a7-3904-41f6-8151-6051852bf1ee service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:50.530122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d66a7-3904-41f6-8151-6051852bf1ee service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:50.530122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d66a7-3904-41f6-8151-6051852bf1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:50.530122 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-c18d66a7-3904-41f6-8151-6051852bf1ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:50.541073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d66a7-3904-41f6-8151-6051852bf1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:24:50.542620 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d66a7-3904-41f6-8151-6051852bf1ee service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:24:50.553541 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1489/2965] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:24:54.285944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:24:54.298605 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58244e9f-24f2-4b26-ba0f-38d0ca0be47e service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:24:54.303920 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:24:54.304521 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:24:54.304800 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:24:54.305073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:24:54.306861 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:24:54.319312 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:24:54.320900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:24:54.331489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1477/2966] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:24:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:25:00.549125 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:25:04.327168 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:26:43.482094 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a1988-2381-4543-bcbb-66cfe22a82a3 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:26:43.482524 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a1988-2381-4543-bcbb-66cfe22a82a3 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:26:43.482640 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a1988-2381-4543-bcbb-66cfe22a82a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:26:43.519855 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-014a1988-2381-4543-bcbb-66cfe22a82a3 None None] MFA Rules not processed for user `422f81200e5645668e123229a6f356d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:26:43.521063 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-014a1988-2381-4543-bcbb-66cfe22a82a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:26:43.570671 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1490/2967] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Aug 24 13:26:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:26:43.580931 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:26:43.589841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208ce55-f9c5-4faf-9f7a-e4a36e4fd463 service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:26:43.593444 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '422f81200e5645668e123229a6f356d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:26:43.594021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:26:43.594295 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:26:43.594559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:26:43.602447 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:26:43.603888 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:26:43.612868 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1478/2968] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:26:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:26:53.565084 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=29 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:26:53.608489 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:26:56.166147 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa097d-9e87-4b9a-8025-92cc71b9e4fa None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:26:56.169764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa097d-9e87-4b9a-8025-92cc71b9e4fa None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:26:56.169764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa097d-9e87-4b9a-8025-92cc71b9e4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:26:56.185216 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-81aa097d-9e87-4b9a-8025-92cc71b9e4fa None None] MFA Rules not processed for user `cc07ef53f88c4a809deab768daff8132`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:26:56.186408 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-81aa097d-9e87-4b9a-8025-92cc71b9e4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:26:56.193329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1491/2969] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 13:26:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:26:57.204662 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:26:57.216816 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c5fc3-690a-4a6c-931e-75eacc090943 service cinder] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:26:57.222871 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9d997-21d1-485b-af5c-a49ff6dae975 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:26:57.223854 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9d997-21d1-485b-af5c-a49ff6dae975 service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:26:57.225283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9d997-21d1-485b-af5c-a49ff6dae975 service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:26:57.225283 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9d997-21d1-485b-af5c-a49ff6dae975 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:26:57.233335 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9d997-21d1-485b-af5c-a49ff6dae975 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:26:57.238946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9d997-21d1-485b-af5c-a49ff6dae975 service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:26:57.249869 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1479/2970] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:26:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:26:58.291120 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d65430-f37b-4933-acf9-27e153e65fed None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:26:58.291653 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d65430-f37b-4933-acf9-27e153e65fed None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:26:58.291769 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d65430-f37b-4933-acf9-27e153e65fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:26:58.311837 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-91d65430-f37b-4933-acf9-27e153e65fed None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:26:58.313196 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-91d65430-f37b-4933-acf9-27e153e65fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:26:58.322509 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1492/2971] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 24 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:26:58.342402 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83e1f-ebf7-420b-99e5-778c569613a4 None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:26:58.342910 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83e1f-ebf7-420b-99e5-778c569613a4 None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:26:58.343156 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83e1f-ebf7-420b-99e5-778c569613a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:26:58.362935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-0ec83e1f-ebf7-420b-99e5-778c569613a4 None None] MFA Rules not processed for user `258097db9804464c81d52fb4aada32ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:26:58.364829 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-0ec83e1f-ebf7-420b-99e5-778c569613a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:26:58.373157 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1480/2972] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Aug 24 13:26:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:26:58.379970 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d65430-f37b-4933-acf9-27e153e65fed None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:26:58.390169 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d65430-f37b-4933-acf9-27e153e65fed None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:26:58.395142 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:26:58.399274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:26:58.399274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:26:58.399274 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:26:58.404602 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:26:58.406278 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:26:58.416268 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1493/2973] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:26:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:27:00.698758 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83e1f-ebf7-420b-99e5-778c569613a4 None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:27:00.708333 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83e1f-ebf7-420b-99e5-778c569613a4 None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:27:00.711944 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:27:00.712540 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:27:00.717536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:27:00.717536 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:27:00.721380 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:27:00.722640 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:27:00.731271 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1481/2974] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:27:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:27:08.411923 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=12 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:27:10.727500 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=15 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:27:16.274827 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:27:16.284714 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35c740-1a1b-4aaa-9161-40e0987821e8 service placement] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:27:16.288375 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96795d3f-d44b-46dd-ba4f-c1784d9fde0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:27:16.288960 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96795d3f-d44b-46dd-ba4f-c1784d9fde0a service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:27:16.289288 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96795d3f-d44b-46dd-ba4f-c1784d9fde0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:27:16.289551 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96795d3f-d44b-46dd-ba4f-c1784d9fde0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:27:16.298748 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96795d3f-d44b-46dd-ba4f-c1784d9fde0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:27:16.300190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96795d3f-d44b-46dd-ba4f-c1784d9fde0a service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:27:16.309784 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1494/2975] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:27:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:27:18.243176 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:27:18.256909 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4a7d0-4b4a-4619-bd5e-fb96ecece0de service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:27:18.263303 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:27:18.263911 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:27:18.264193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:27:18.264463 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:27:18.266241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:27:18.279003 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:27:18.280563 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:27:18.303075 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1482/2976] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:27:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:27:26.305502 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=6 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:27:28.287091 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:28:49.014240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4526e-9f4a-46ed-8927-6147b1b53336 None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:28:49.014847 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4526e-9f4a-46ed-8927-6147b1b53336 None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:28:49.014992 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4526e-9f4a-46ed-8927-6147b1b53336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:28:49.047075 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-2db4526e-9f4a-46ed-8927-6147b1b53336 None None] MFA Rules not processed for user `639c0cb750744031b47ba3e48170cce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:28:49.048282 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-2db4526e-9f4a-46ed-8927-6147b1b53336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:28:49.054764 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1495/2977] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Aug 24 13:28:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:28:49.068748 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:28:49.077900 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205915-5918-4d0c-b647-8c068d461937 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:28:49.082021 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2aeca-a588-43e5-a36a-14df6f8756a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:28:49.082596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2aeca-a588-43e5-a36a-14df6f8756a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:28:49.082902 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2aeca-a588-43e5-a36a-14df6f8756a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:28:49.083170 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2aeca-a588-43e5-a36a-14df6f8756a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:28:49.089843 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2aeca-a588-43e5-a36a-14df6f8756a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:28:49.091275 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2aeca-a588-43e5-a36a-14df6f8756a6 service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:28:49.098794 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1483/2978] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:28:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:28:59.042389 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=8 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:28:59.095520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=6 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:07.881093 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4295b-0beb-4eaa-9666-832148f4e30c None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:07.894329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4295b-0beb-4eaa-9666-832148f4e30c None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:07.894329 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4295b-0beb-4eaa-9666-832148f4e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:07.938788 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-59f4295b-0beb-4eaa-9666-832148f4e30c None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:07.940015 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-59f4295b-0beb-4eaa-9666-832148f4e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:07.970260 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1496/2979] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 13:29:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:07.986952 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864951d-6fdb-4e37-9a21-1ee17b962abb None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:07.987422 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864951d-6fdb-4e37-9a21-1ee17b962abb None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:07.987538 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864951d-6fdb-4e37-9a21-1ee17b962abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:08.007541 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-2864951d-6fdb-4e37-9a21-1ee17b962abb None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:08.008814 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-2864951d-6fdb-4e37-9a21-1ee17b962abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:08.018093 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1484/2980] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 13:29:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:08.025436 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4295b-0beb-4eaa-9666-832148f4e30c None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:08.037512 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4295b-0beb-4eaa-9666-832148f4e30c None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:08.045606 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:08.046226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:08.046508 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:08.046840 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:08.058881 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:08.060873 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:08.075293 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1497/2981] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:29:08.337358 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12a500-8d7f-489a-a06c-00ed5db5dc3c None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:08.337647 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12a500-8d7f-489a-a06c-00ed5db5dc3c None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:08.337946 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12a500-8d7f-489a-a06c-00ed5db5dc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:08.358263 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-8f12a500-8d7f-489a-a06c-00ed5db5dc3c None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:08.359525 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-8f12a500-8d7f-489a-a06c-00ed5db5dc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:08.369511 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1485/2982] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 13:29:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:08.381138 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:08.390629 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b1616-3ce8-4240-9a6a-b8edb7c9132d service nova] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:08.394989 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1b6a8-c44a-499d-ab94-98c1ff2fd0bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:08.395582 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b6a8-c44a-499d-ab94-98c1ff2fd0bd service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:08.395862 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b6a8-c44a-499d-ab94-98c1ff2fd0bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:08.396129 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b6a8-c44a-499d-ab94-98c1ff2fd0bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:08.407048 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1b6a8-c44a-499d-ab94-98c1ff2fd0bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:08.408544 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1b6a8-c44a-499d-ab94-98c1ff2fd0bd service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:08.421108 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1498/2983] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:29:09.597048 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee973c-402a-4e43-b13d-6ad5eb1401cc None None] REQUEST_METHOD: `POST` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:09.597613 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee973c-402a-4e43-b13d-6ad5eb1401cc None None] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:09.597721 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee973c-402a-4e43-b13d-6ad5eb1401cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:09.617898 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.auth.core [None req-43ee973c-402a-4e43-b13d-6ad5eb1401cc None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81681) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:09.619145 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-43ee973c-402a-4e43-b13d-6ad5eb1401cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:09.629177 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1486/2984] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 13:29:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:09.634880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aca9e9-6f5c-40c8-a24e-7f69a29304ac None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:09.635358 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aca9e9-6f5c-40c8-a24e-7f69a29304ac None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:09.635464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aca9e9-6f5c-40c8-a24e-7f69a29304ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:09.654442 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-48aca9e9-6f5c-40c8-a24e-7f69a29304ac None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:09.655690 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-48aca9e9-6f5c-40c8-a24e-7f69a29304ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:09.668141 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1499/2985] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 13:29:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:09.681445 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee973c-402a-4e43-b13d-6ad5eb1401cc None None] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:09.691241 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee973c-402a-4e43-b13d-6ad5eb1401cc None None] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:09.696856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:09.696856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:09.696856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:09.696856 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:09.705622 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:09.707094 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:09.718528 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1487/2986] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:29:16.480420 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400701-232c-44d4-8989-28836740b41a None None] REQUEST_METHOD: `POST` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:16.480936 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400701-232c-44d4-8989-28836740b41a None None] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:16.481194 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400701-232c-44d4-8989-28836740b41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:16.502619 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.auth.core [None req-78400701-232c-44d4-8989-28836740b41a None None] MFA Rules not processed for user `0ed16fea638a45938eee8cdf234333f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81680) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:16.503980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-78400701-232c-44d4-8989-28836740b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:16.513561 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1500/2987] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:16.527364 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:16.539953 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7bc6a-f6af-4c30-99b5-f95772ab150c service neutron] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:16.547448 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ed16fea638a45938eee8cdf234333f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:16.550243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:16.550243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:16.550243 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:16.558643 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:16.560415 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:16.572614 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1488/2988] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:29:23.671720 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400701-232c-44d4-8989-28836740b41a None None] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:23.681469 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400701-232c-44d4-8989-28836740b41a None None] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:23.685076 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639c0cb750744031b47ba3e48170cce1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:23.685665 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:23.685948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:23.686220 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:23.747099 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:23.748531 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:23.759209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1501/2989] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:29:26.568569 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=21 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:27.120109 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:27.132116 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a77b-0552-48d2-9aaa-37ca7518fa95 service nova] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:27.136971 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:27.137559 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] REQUEST_METHOD: `GET` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:27.137836 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:27.138107 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:27.139866 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:27.152367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:27.154030 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:27.164002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1489/2990] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:29:30.900252 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:30.905361 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=70 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:30.910987 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ed8e5-b67a-432a-b042-b492e1dbd44d service neutron] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:30.915743 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:30.916368 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:30.916644 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:30.916914 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] PATH_INFO: `/v3/users/cc07ef53f88c4a809deab768daff8132` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:30.918029 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc07ef53f88c4a809deab768daff8132)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:30.919980 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:31.073618 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc07ef53f88c4a809deab768daff8132'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.074197 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc07ef53f88c4a809deab768daff8132'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.121240 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc07ef53f88c4a809deab768daff8132'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.134259 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc07ef53f88c4a809deab768daff8132'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.134917 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc07ef53f88c4a809deab768daff8132'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.147783 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1502/2991] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:29:30 2022] DELETE /identity/v3/users/cc07ef53f88c4a809deab768daff8132 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:29:31.397579 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:31.408376 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572306d-81a7-4c3f-a8da-0c2c5e4afceb service placement] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:31.474430 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:31.475115 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:31.475397 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:31.475673 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] PATH_INFO: `/v3/projects/e4ab80b71d1f4f06b20c597689ae8a2e` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:31.476838 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4ab80b71d1f4f06b20c597689ae8a2e)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:31.478891 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:31.590841 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4ab80b71d1f4f06b20c597689ae8a2e', 'request_id': 'req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.597596 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4ab80b71d1f4f06b20c597689ae8a2e', 'request_id': 'req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.603139 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4ab80b71d1f4f06b20c597689ae8a2e', 'request_id': 'req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.669002 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] The token cache is being invalidate because project e4ab80b71d1f4f06b20c597689ae8a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:29:31.669228 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.669849 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.671367 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1490/2992] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:29:31 2022] DELETE /identity/v3/projects/e4ab80b71d1f4f06b20c597689ae8a2e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:29:31.675981 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:31.677398 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:31.690159 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4d96-14ec-4de6-a10e-a73505eb443b admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:31.745598 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:31.746209 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:31.746481 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:31.746854 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] PATH_INFO: `/v3/users/8df5b9f369bf491b9ef61bc45ad5cd89` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:31.747961 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df5b9f369bf491b9ef61bc45ad5cd89)` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:31.749880 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:31.949464 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df5b9f369bf491b9ef61bc45ad5cd89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:31.950111 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df5b9f369bf491b9ef61bc45ad5cd89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.024154 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df5b9f369bf491b9ef61bc45ad5cd89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.037297 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df5b9f369bf491b9ef61bc45ad5cd89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.037948 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.notifications [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df5b9f369bf491b9ef61bc45ad5cd89'} {{(pid=81680) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.042565 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:32.050613 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1503/2993] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 13:29:31 2022] DELETE /identity/v3/users/8df5b9f369bf491b9ef61bc45ad5cd89 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:29:32.334935 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Authenticating user token {{(pid=81681) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:32.336356 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.fernet_utils [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81681) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:32.349193 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfcd3e-1844-48cb-b0ee-c8e99025bd17 admin admin] Validating token access rules against request {{(pid=81681) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:32.406987 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad9be8700f45469d8fb5f44fbb2eeb2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cfd603abe4c44b2a5789dbc4d8a6f2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81681) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:32.407851 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:32.408311 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:32.408733 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] PATH_INFO: `/v3/projects/7461b8d063104ad0b8da1e8bdf9ee73f` {{(pid=81681) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:32.410449 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7461b8d063104ad0b8da1e8bdf9ee73f)` {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:32.412442 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] RBAC: Authorization granted {{(pid=81681) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:32.434395 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:32.533362 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7461b8d063104ad0b8da1e8bdf9ee73f', 'request_id': 'req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.539218 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7461b8d063104ad0b8da1e8bdf9ee73f', 'request_id': 'req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.544612 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7461b8d063104ad0b8da1e8bdf9ee73f', 'request_id': 'req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d'} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.612073 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] The token cache is being invalidate because project 7461b8d063104ad0b8da1e8bdf9ee73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81681) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:29:32.612412 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.613029 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG keystone.notifications [None req-d62eac58-91dc-47f4-b33e-344f3fe5ea9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81681) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:29:32.614488 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: [pid: 81681|app: 0|req: 1491/2994] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 13:29:32 2022] DELETE /identity/v3/projects/7461b8d063104ad0b8da1e8bdf9ee73f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:29:42.338699 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=3 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:42.618520 dv-cinder-848899-3-iscsi devstack@keystone.service[81681]: DEBUG dbcounter [-] [81681] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:04.282779 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Authenticating user token {{(pid=81680) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:04.284190 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:04.435792 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c112b5d-549e-4a7f-a1ca-6b77a80c6958 admin admin] Validating token access rules against request {{(pid=81680) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:04.494173 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c66c24-4488-41ed-9368-c5be0495b5d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258097db9804464c81d52fb4aada32ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aff3a8a82b04c4798539c93ca7ee2b4', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81680) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:04.494794 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c66c24-4488-41ed-9368-c5be0495b5d1 service placement] REQUEST_METHOD: `GET` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:04.495072 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c66c24-4488-41ed-9368-c5be0495b5d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:04.495350 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c66c24-4488-41ed-9368-c5be0495b5d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81680) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:04.497073 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.fernet_utils [None req-54c66c24-4488-41ed-9368-c5be0495b5d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81680) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:04.508226 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c66c24-4488-41ed-9368-c5be0495b5d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:04.509805 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c66c24-4488-41ed-9368-c5be0495b5d1 service placement] RBAC: Authorization granted {{(pid=81680) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:04.519506 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: [pid: 81680|app: 0|req: 1504/2995] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:14.515520 dv-cinder-848899-3-iscsi devstack@keystone.service[81680]: DEBUG dbcounter [-] [81680] Writing DB stats keystone:SELECT=89 {{(pid=81680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}